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ABSTRACT: 

Advances  in  graphic  display  technologies  have  made  virtual  reality  (VR)  and  scientific 
visualization  applications  accessible  to  a  wide  user  population.  Unfortunately,  few  human 
interfaee  tools  exist  to  allow  users  to  interact  naturally  with  these  powerful  graphical 
environments.  To  address  this  need,  Immersion  Corporation  has  developed  a  user  interface 
mechanism  to  allow  natural  manual  interaction  with  3-D  environments  which  provides  realistic 
force  feedback  to  the  user.  This  haptic  display  methodology  combines  high  fidelity,  low  cost, 
and  inherent  safety  to  allow  force  reflection  technology  to  become  commercially  feasible. 

The  long  term  objective  is  to  produce  a  3-D  haptic  interface  for  virtual  environments.  Phase  I 
focused  on  producing  one-dimensional  haptic  interface  hardware  and  incorporating  this 
technology  into  a  real  world  VR  application.  Immersion  and  the  Ohio  Supercomputer  Center 
have  worked  together  to  produce  a  virtual  simulation  of  epidural  analgesia,  a  medical  procedure 
that  requires  delicate  needle  insertions  into  the  spinal  column.  The  resulting  VR  simulation  is  so 
realistic  in  look  and  feel,  it  can  actually  be  used  as  a  training  environment  to  teach  doctors  to 
perform  the  dexterous  manual  procedure,  allowing  them  to  learn  manual  technique  and  explore  the 
associated  physical  sensations  without  the  risks  or  costs  associated  with  using  real  biological . 

Our  six-degree-of-ffeedom  haptic  display  has  immediate  commercial  applications  in  the  medical 
simulation  and  scientific  visualization  fields.  The  technology  is  also  applicable  to  every  form  of 
computer  interface,  including  computer-aided  design  packages,  scientific  modeling,  entertainment, 
and  educational  tutorials. 
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A.  Identification  and  Discussion  of  the  Effort 

A.1  Background:  New  Opportunity  for  Haptic  Display  Development 

In  recent  years,  advances  in  graphic  display  technology  have  made  possible  the  rapid  and 
economical  generation  of  high  fidelity,  three-dimensional  images.  From  virtual  reality  to 
scientific  visualization,  the  potential  applications  of  high  fidelity  3-D  computer  environments 
span  the  gamut  from  military  training  to  surgical  simulation.  Unfortunately,  the  recent 
technological  advances  in  visual  information  display  have  not  been  adequately  supported  by 
innovative  means  of  interfacing  humans  to  computers.  Very  little  progress  has  been  made 
towards  developing  human  interface  tools  that  allow  users  to  perform  natural  manual  interactions 
with  3-D  computer  environments.  Just  as  the  computer  "mouse"  empowers  users  to  perform 
natural  manual  interactions  with  2D  environments,  new  human  interface  technologies  are  now 
needed  to  empower  users  to  interact  naturally  with  3-D  virtual  environments. 

To  allow  users  to  execute  natural  manual  interaction  with  3-D  virtual  environments,  two  primary 
issues  must  be  addressed  when  developing  effective  human  interface  technologies.  These  two 
issues  can  be  classified  simply  as  manual  input  and  manual  feedback  respectively.  The  first  of 
these,  manual  input,  requires  the  development  of  human  interface  technologies  that  allow  users 
to  execute  natural  and  comfortable  gestures  in  3-D  space  with  minimal  fatigue  and  optimal 
dexterity.  Immersion  Corporation  produces  a  number  of  products  for  3-D  input  and  manual 
control.  For  cursor  control  and  arbitrary  manual  input  tasks,  we  sell  the  Immersion  PROBE,  a 
six-degree-of-freedom  tracking  device  which  uses  an  articulated  arm  to  track  and  support  motion. 

The  second  issue  which  needs  to  be  addressed  is  manual  feedback.  Known  as  "haptic  display" 
or  "force  feedback",  the  ability  to  provide  users  with  realistic  "feel"  information  about  the 
virtual  environment  they  are  interacting  with  is  one  of  the  most  challenging  human  interface 
problems  facing  researchers  today.  To  date,  only  a  handful  of  human  interface  devices  have  been 
developed  which  provide  users  of  virtual  environments  with  force  feedback.  Without  exception, 
these  devices  have  been  developed  as  research  tools,  with  price  tags  which  put  them  out  of  the 
reach  of  the  user  population.  As  a  result,  the  commercialization  of  haptic  display  technology  has 
been  negligible.  Potential  user  s  such  as  engineers,  designers,  scientists,  and  medical  professionals 
who  could  benefit  greatly  from  addition  of  force  feedback  to  their  virtual  simulations  have  not 
had  access  to  such  technologies.  To  address  this  need.  Immersion  Corporation  intends  to 
develop  the  first  commercially  viable,  high  performance,  haptic  interface  tool  for  manual 
interaction  with  3-D  virtual  environments. 

A.2  The  Nature  of  the  Project 

If  this  project  continues  through  Phase  II,  we  plan  to  develop  a  six-degree-of-freedom  user 
interface  which  will  not  only  be  an  input  device  for  commanding  position  and  orientation  of  a 
cursor  in  a  3-D  information  space,  but  will  also  be  an  output  device  which  reflects  physical 
forces  back  to  the  user.  Such  a  haptic  display  interface  would  be  capable  of  presenting  physical 
properties  of  the  3-D  information  space  by  representing  surfaces  and  contours  with  actual 
physical  forces.  Although  a  handful  of  force-reflecting  devices  eire  currently  available  to  large 
research  laboratories,  the  typical  approach  to  force  reflection  has  resulted  in  interface  hardware 
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which  is  so  complex  and  expensive  that  it  can  only  be  used  by  trained  personnel  in  elite  research 
facilities  devoted  to  studying  force  reflection.  These  devices  are  therefore  out  of  reach  for  the  end 
users  who  would  benefit  most  from  such  haptic  display  tools  --  scientists,  engineers,  medical 
professionals,  and  designers. 

We  at  Immersion  Human  Interface  Corporation  have  taken  on  the  challenge  of  bringing  force 
reflection  technology  out  of  the  research  lab  and  onto  the  desks  of  scientists  and  engineers.  To 
accomplish  this  important  step,  we  have  developed  a  hybrid  approach  to  force  reflection  which 
will  turn  haptic  display  hardware  into  economically  feasible  commercial  products  for  mainstream 
use.  The  details  of  our  hybrid  haptic  display  methodology  will  be  discussed  in  detail  later  in  this 
document. 

A.3  Phase  I  Overview 

Immersion  Corp.  used  Phase  I  to  tackle  a  subset  of  the  overall  technical  challenge  by  developing  a 
single-degree-of-freedom  haptic  display  for  one  dimensional  interaction.  This  single  degree  of 
freedom  system,  while  much  simpler  than  the  ultimate  6-degree-of-freedom  system  proposed  for 
Phase  II,  has  afforded  an  excellent  feasibility  study  for  the  final  system  by  allowing  us  to  test  and 
refine  our  innovative  approach  to  force  reflection  before  proceeding  to  Phase  II.  The  Phase  I 
project  has  also  resulted  in  an  immediately  applicable  haptic  interface  for  an  important  real  world 
application.  We  are  encouraged  by  the  success  of  our  Phase  I  single  degree-of-freedom  haptic 
devices,  which  not  only  demonstrate  the  prowess  of  our  haptic  display  methodology,  but  also 
demonstrate  the  commercial  potential  of  force  reflection  technology  in  general. 

This  powerful  real-world  application  mentioned  above  is  a  virtual  reality  system  developed  to 
train  doctors  to  perform  a  delicate  one-dimensional  medical  procedure  known  as  epidural 
analgesia.  This  medical  simulator  was  developed  as  a  joint  effort  between  Immersion 
Corporation  and  experts  in  the  field  of  medical  simulation  and  training  at  Ohio  Supercomputer 
Center.  In  order  to  create  an  effective  simulation,  we  developed  a  single-degree-of-freedom  haptic 
interface  which  is  optimized  to  display  the  force  information  associated  with  such  medical 
procedures.  The  end  product  of  our  Phase  I  effort,  the  Virtual  Epidural  Simulator,  acts  as  a  very 
powerful  proof  of  concept  of  our  force  reflection  technology,  not  to  mention  an  immediately 
applicable  technology.  Extending  this  technology  to  three-dimensional  space  in  Phase  II  will 
allow  us  to  simulate  more  complex  medical  procedures  such  as  laparoscopic  surgery. 


Virtual  Epidural  Simulator 

(A  single-degree-of-freedom  virtual  simulation  with  force  feedback); 

Epidural  analgesia  is  one  of  the  most  frequently  used  techniques  for  the  relief  of  pain 
during  surgery.  In  most  hospitals  that  have  an  obstetric  anesthesia  service,  epidural 
analgesia  is  the  most  prominently  used  anesthesia  technique  for  vaginal  childbirth  and 
cesarean  section.  The  procedure  involves  the  injection  of  a  local  anesthetic  or  an  opiod 
into  the  epidural  space  of  the  spinal  column.  Although  a  single  degree-of-freedom  task,  it 
is  a  delicate  manual  operation  which  requires  the  surgeon  to  guide  a  catheter  into  the 
epidural  space  using  only  haptic  cues  to  guide  the  needle.  By  feeling  the  needle  passing 
though  various  tissues,  the  doctor  needs  to  maneuver  it  into  the  correct  space  without 
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damaging  the  spinal  cord  in  the  process. 

One  major  problem  with  this  procedure  is  that  there  currently  exists  NO  way  of  training 
residents  on  this  dangerous  manual  task  without  actually  having  them  learn  on  live 
patients.  By  creating  a  virtual  simulation  of  this  procedure,  Immersion  Corporation  and 
Ohio  Supercomputer  Center  have  produced  an  invaluable  tool  for  training  doctors  to 
perform  this  vital  task.  The  simulator  includes  a  force-reflecting  interface  which  generates 
the  appropriate  haptic  cues  associated  with  needle  penetration  of  the  various  layers  of 
muscle  and  tendon.  The  Virtual  Epidural  Simulator  provides  a  trainee  with  a  believable, 
non-threatening  environment  which  encourages  the  free  exploration  of  the  many  variances 
confronted  in  administering  a  lumbar  epidural.  Such  interaction  and  exploration  with  a 
virtual  simulation  should  increase  the  residents  understanding  of  the  anatomy  and 
intricacies  encountered  in  the  epidural  procedure.  It  is  hypothesized  that  such  a  training 
environment  will  reduce  resident  anxiety  and  limit  the  number  of  trials  required  to  learn  to 
successfully  deliver  regional  anesthesia  through  the  epidural  method. 

In  the  Phase  I  effort.  Immersion  Corporation  focused  on  the  development  of  the  force 
reflection  hardware  required  for  the  virtual  simulation  of  the  epidural  procedure,  while 
Ohio  Supercomputer  Center  developed  the  graphic  simulation  of  the  procedure  by 
providing  high  level  software  support  and  by  providing  direct  exposure  to  medical 
personnel  who  can  provide  performance  evaluations  of  the  hardware.  Together 
Immersion  Corp.  and  Ohio  Supercomputer  Center  formed  an  effective  team  to  develop 
and  evaluate  the  single-degree-of-freedom  haptic  interface  system. 

B.  Technical  Objectives  of  the  Effort 

Shown  in  bold  are  the  technical  objectives  as  stated  in  the  Phase  I  proposal. 

Design  and  construct  a  single-degree-of-freedom  haptic  display,  complete  with  sensor 
elements  and  force  reflecting  elements  that  can  generate  controllable  passive  resistance 
and  controllable  active  force  output.  This  system  must  have  the  fidelity  required  to 
accurately  reproduce  the  haptic  sensations  associated  with  medical  procedures  such  as 
epidural  analgesia. 

The  primary  objective  of  the  Phase  I  proposal  was  to  develop  a  one-degree-of-freedom  haptic 
display  that  can  be  extended  to  a  full  six-degree-of-freedom  haptic  display  in  Phase  II.  The 
one  degree  of  freedom  haptic  display  was  targeted  at  simulating  the  feel  of  a  specific  medical 
procedure,  epidural  analgesia. 

Before  designing  and  building  the  device,  testing  was  performed  with  an  instrumented  probe 
to  discover  the  bandwidth  and  magnitude  of  haptic  sensations  required  by  medical  procedures 
and  by  haptic  interactions  in  general.  A  piece  of  hardware  was  constructed  which  combined  a 
six-degree-of-freedom  tracking  probe  cormected  to  an  epidural  needle  that  was  instrumented 
with  a  high  fidelity  force  sensor.  Using  this  specialized  hardware,  skilled  doctors  could 
perform  epidural  procedures  on  cadaver  specimens  while  position  and  force  data  was 
recorded.  By  recording  the  three  dimensional  location  of  the  needle  and  the  forces  generated 
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on  the  needle  during  the  insertion  proeedure,  we  were  able  reconstruct  the  force  profile 
associated  with  the  epidural  procedure.  Extensive  testing  was  performed  on  cadaver 
specimens  to  quantify  the  force  profile  and  frequency  content  of  haptic  sensations 
representative  of  epidural  needle  insertion  into  the  spinal  column.  Analysis  of  this  data  was 
extremely  useful  in  guiding  the  design  of  the  virtual  reality  system  to  simulate  such  haptic 
information. 

In  addition  to  the  direct  force/position  measurements  during  actual  manual  epidural 
procedures,  CT  scan  data  recorded  taken  from  the  operative  site  of  the  cadaver  specimen. 
Analysis  was  performed  to  assess  if  force  profile  information  could  be  correlated  with  the  CT 
scan  data.  The  ultimate  goal  is  to  establish  a  methodology  where  virtual  reality  simulations  of 
haptic  sensations  can  be  derived  directly  from  volumetric  data  such  as  CT  or  MRI  scans. 

Once  testing  was  complete,  our  next  step  was  to  construct  two  versions  of  a  simple  single 
degree-of-ffeedom  force  reflecting  device.  The  first  one  of  these  was  a  passive  device,  using 
magnetic  particle  brakes  to  provide  resistance  to  motion.  A  second  device  was  constructed  as 
well  which  incorporated  a  linear  capstan  drive  mechanism  and  an  active  servo  motor  for 
actuation.  These  two  devices  were  constructed  so  that  active  and  passive  force  feedback 
techniques  could  be  compared  and  contrasted. 


Develop  integrated  Virtual  Epidural  Simulator  and  Assess  Effectiveness 

The  hardware  mentioned  above  was  sent  to  OSC  for  integration  into  a  single  degree-of- 
freedom  surgical  simulation.  The  device  using  servo  motors  was  used  to  construct  the  Virtual 
Epidural  Simulator.  For  the  simulator,  OSC  developed  high-level  software,  including 
graphics,  user-interface,  and  simulation  capabilities  to  integrate  the  one-degree-of-freedom 
haptic  device  into  their  medical  simulation  application.  The  resulting  full  system  of  hardware 
and  software  provides  a  fully  functional  virtual  simulation  of  the  epidural  procedure.  This 
simulation  provides  the  appropriate  haptic  sensations  associated  with  needle  penetration  of 
the  various  types  of  tissues  encountered  during  such  a  procedure. 

Ongoing  tests  with  medical  practitioners  has  provided  data  on  performance  and  effectiveness 
for  specific  haptic  sensations. 

The  Phase  I  effort  resulted  in  a  complete  simulation  system  which  combines  a  convincing 
haptic  experience  with  an  intricate  simulation  of  the  medical  procedures.  The  success  of 
Pheise  I  clearly  warrants  a  Phase  II  effort  by  demonstrating  the  applicability  of  our  force 
feedback  technologies  to  medical  simulation.  In  addition  to  demonstrating  the  general 
effectiveness  of  adding  force  feedback  technologies  to  virtual  reality  simulations,  this  Phase  I 
effort  demonstrated  the  kinematic  limitations  of  the  Phase  I  single-degree-of-freedom  device 
and  thus  strongly  suggests  that  the  full  six-degree-of-freedom  device  should  be  pursued  in 
Phase  II. 

Develop  a  design  specifications  document  which  enumerates  the  optimal  parameters  for  a 
stylus-based  haptic  display.  Such  a  document  will  list  appropriate  design  parameters 
such  as  size,  mass,  inertia,  allowable  friction,  balance,  range  of  motion,  and  resolution 
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for  a  mechanical  system  that  supports  haptic  display.  Such  a  document  will  also 

enumerate  the  requirements  for  a  hybrid  haptic  display  such  as  bandwidth,  maximum 

resistance  forces,  and  latency  requirements. 

One  important  step  that  is  often  overlooked  in  the  design  of  human  interface  hardware  is 
quantifying  the  perceptual  requirements  of  the  user.  Our  final  objective  is  not  simply  to 
produce  a  haptic  interface  tool  with  technical  merit,  but  also  to  produce  one  that  conforms 
well  to  appropriate  specifications  for  a  system  with  a  human  in  the  loop.  Since  there  are  no 
widely  accepted  design  specifications  for  systems  that  support  manual  interaction  within 
immersive  computer  environments,  an  important  part  of  Phase  I  has  been  to  determine  some 
appropriate  specifications  for  the  human-computer  system  we  intend  to  complete  in  Phase 
II.  The  fidelity  of  a  force  feedback  system  depends  on  the  accuracy  of  the  sensors,  the 
processing  speed  of  the  central  computer,  the  performance  of  the  actuators,  and  the 
transparency  of  the  mechanical  transmission.  The  contribution  that  each  of  the  above  sub¬ 
systems  has  upon  system  performance  is  discussed  in  great  detail  in  Appendix  A:  Design 
Specifieations  Document. 

C.  Phase  I  Statement  of  Work  Completed 

Phase  I  efforts  have  proceeded  along  three  fronts: 

1 .  Immersion  Corporation  developed  force  feedback  hardware  using  both  passive  and  active 
technologies  as  laid  out  in  our  original  proposal.  This  hardware  was  sent  to  OSC  for 
integration  into  the  full  virtual  reality  medical  simulation  platform.  In  addition.  Immersion 
constructed  other  prototype  hardware  for  in-house  evaluation. 

2.  Immersion  analyzed  the  performance  of  the  force  feedback  hardware  prototypes  and 
analyzed  the  data  resulting  from  OSC’s  testing  efforts. 

3.  OSC  has  been  exploring  simulation  techniques,  including  graphics,  interaction  techniques, 
MRI  and  CT  data  processing,  and  haptic  presentations. 

These  efforts  are  described  in  detail  in  the  following  pages. 


C.l  Hardware  Developed 

First  Prototypes  of  1  degree-of-freedom  haptic  interface 

Single  Degree-of-Freedom  Active  System 

Force  and  position  sensing  development  hardware  is  needed  to  devise  models  to  be  used 
in  the  position  sensing  and  force  producing  product  hardware.  This  hardware  is  described 
below  and  a  full  characterization  of  the  hardware  is  presented  in  Appendix  C. 
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Haptic  Sensing  Hardware 
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Figure  C-l:  Hardware 

Figure  C-l  shows  both  the  haptic  sensing  hardware  and  haptic  feedback  hardware.  The 
haptic  sensing  hardware  is  essentially  an  instrumented  epidural  needle.  The  epidural 
needle  has  been  mounted  on  an  extra  long  lightweight  handle,  which  is  constrained  to 
linear  motion  without  twist  by  a  set  of  high  quality  bearings.  A  small  strain  gage- 
outfitted  load  cell  which  has  been  Electro-discharge-machined  out  of  aluminum  comprises 
the  force  sensor.  The  amplification  circuitry  is  located  right  next  on  the  load  cell  on 
surface  moxmt  printed  circuit  board.  An  in-house  design  16-bit  A/D  converter  and  a  low- 
cost  digital  I/O  card  allow  for  acquisition  of  the  force  signal  by  a  486DX2-66  PC. 

Finally,  a  1024  count/revolution  rotary  encoder  with  a  fiiction  drive  wheel  on  the  needle 
handle  is  responsible  for  transducing  the  position  of  the  needle. 

The  hardware  for  the  haptic  feedback  product  is  made  up  of  a  linear  slide,  motorized 
through  a  capstan  drive  by  a  low-inertia  precision  motor.  A  highly  flexible  steel  cable 
takes  3  turns  around  a  screw  pulley  on  the  motor.  For  simplicity  and  low  cost,  we  have 
decided  not  to  include  a  force  sensor  in  the  final  product.  Therefore,  measured  force  will 
not  be  available  for  use  in  a  control  law  or  table  lookup  scheme.  An  encoder  is  again 
responsible  for  tracking  position  of  the  slide.  To  minimize  inertia,  the  rod  itself  moves 
and  the  platform  is  fixed. 

The  haptic  sensing  and  feedback  hardware  is  used  in  three  configurations.  A  sensing 
hardware  is  used  alone  with  tissue  substitute  samples  to  generate  simulation  models.  The 
sensing  hardware  is  then  used  together  with  the  feedback  hardware  to  test  candidate 
models.  Finally,  the  feedback  hardware  alone  will  constitute  the  simulator. 

Single  Degree-of-Freedom  Needle  Insertion  Simulator 

Immersion  produced  a  single  degree-of-ffeedom  needle  insertion  simulator  to  provide  OSC 
with  a  start  on  their  simulation  efforts.  The  device,  shown  below  in  Figure  C-2  uses  a 
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passive  resistance  device  to  apply  resistive  forces  to  a  needle.  The  hardware  is  accessed 
by  the  computer  through  a  standard  serial  port,  providing  relatively  low-bandwidth  I/O. 

The  device  allows  a  needle  to  be  inserted  and  withdrawn  into  a  block  which  can  be 
programmed  to  provide  resistive  forces  upon  demand  to  simulate  forces.  The  results 
obtained  by  OSC  can  be  found  in  section  C.3. 


Needle  Insertion  Simulator  Device  Being  Used  at  OSC 

Figure  C-2:  Passive  Single  Degree-of-Freedom  Needle  Insertion  Simulator 


Instrumented  needle  to  send  to  OSC 

A  5  pound  single-axis  tension/compression  load-cell  (model  ALD-MINI-UTC  0.5”  from 
A.L.  Design  in  Buffalo,  NY)  was  built  into  a  needle  sent  to  us  by  OSC.  The  needle  was 
cut  along  a  central  cross-section  and  the  sensor  was  inserted  to  measure  forces  along  the 
axis  of  the  needle.  The  instrumented  needle  was  then  attached  to  the  Immersion  Probe 
(see  next  section)  to  track  its  position  along  with  the  needle  forces. 

Immersion  Probe’’’^ 

The  Immersion  Probe  is  a  six  degree-of-ffeedom  articulated  arm  which  can  be  used  to 
track  the  position  and  orientation  of  a  stylus  in  3D  space.  Since  all  six  degrees-of- 
freedom  are  reported,  the  device  can  be  used  as  a  general-purpose  manipulation  tool. 
OSC  used  the  device  to  report  the  position  and  orientation  of  the  instrumented  needle 
described  above  in  real  time.  See  Figure  C-3  for  a  look  at  the  procedure. 


9 


Figure  C-3:  Immersion  Probe  (left)  &  Immersion  Probe  being  used 
with  the  Instrumented  Needle  (right) 


Second  Prototype:  1  Degree-of-Freedom  Impulse  Engine 

To  provide  OSC  with  a  high  performance  single  degree-of-freedom  force  reflecting 
system,  Immersion  designers  assembled  the  second  prototype,  which  consists  of  a  linear 
capstan  drive  mechanism  coupled  to  a  servo  motor.  Electronics  are  interfaced  to  the 
computer  via  the  PC  bus.  The  following  subsections  outline  the  capstan  design  and  the 
electronics  design. 

The  Linear  Capstan  Design: 

The  basic  linear  axis  consisted  of  a  precision  ground  shaft  which  slid  prismatically 
through  a  Teflon  coated  bearing  surface.  A  high  tension  aircraft  cable  was  fixed  to  either 
end  of  the  linear  shaft  and  was  wound  around  a  small  diameter  capstan  pulley .  This 
pulley  was  fixed  to  the  drive  motor  and  thus  power  to  the  motor  applied  linear  force  to 
the  shaft.  An  optical  encoder  was  mounted  on  the  servo  motor  to  monitor  the  position  of 
the  linear  axis.  This  configuration  achieved  the  required  force  output  and  required  range  of 
motion  while  maintaining  low  fiiction,  high  stiffness,  and  negligible  backlash.  The 
configuration  allowed  the  linear  drive  to  be  tracked  with  the  required  accuracy  in  position. 
The  final  linear  drive  system,  as  shown  below  in  Figure  C-4,  resulted  in  a  smooth  drive 
mechanism  which  could  produce  high  bandwidth  haptic  sensations  and  introduced 
minimal  noise. 
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Figure  C-4;  Linear  Capstan  Design:  Actual  Hardware  (left),  Drawing  (right) 


Electronics:  Interface  and  Drive  Circuits 


We  have  developed  an  electronics  architecture  which  allows  a  standard  PC  computer  to 
interface  with  our  force  feedback  hardware.  This  architecture,  shown  in  Figure  C-5,  can 
be  broken  up  into  a  number  of  subsystems  which  include  an  ISA  CARD,  a  POWER 
AMPLIFIER,  and  the  SENSOR/ACTUATOR  subsystem.  The  ISA  CARD  is  the  central 
component  which  allows  the  other  subsystems  to  interface  with  the  mother  board  of  the 
host  computer.  The  POWER  AMPLIFIER  provides  power  to  the  actuators,  based  on 
the  low  level  force  command  signals  produced  by  the  ISA  card.  These  power  amplifiers 
use  their  own  power  supply  to  remain  independent  of  the  host  computer  and  to  provide 
maximum  performance.  Finally,  the  sensors  and  actuators  in  the  SENSOR/ACTUATOR 
subsystem  provide  the  position  input  and  force  output  respectively  for  the  user  interface. 


Sensor  Data 


Force  Commands 


Power  Amplifier  Electronics 


Sensors/Actuators 


Actuator  Power  Signals 


Power 

Supply 


Figure  C-5,  Electronics  Design  Overview 


For  a  detailed  description  of  each  of  these  subsystems,  see  Appendix  E. 
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C.2  Analytical  Studies  Performed  at  Immersion 

Evaluation  of  typical  forces  in  haptic  encounters 

In  an  earlier  effort  at  Immersion,  psychophysical  testing  was  performed  on  human 
subjects  to  reveal  the  minimum  performance  requirements  of  a  stylus-based  haptic 
interface.  The  goal  was  to  use  the  limitations  in  human  perceptual  abilities  as  a  governing 
factor  in  determining  the  maximum  hardware  requirements.  Physical  bandwidth  and 
maximum  force  output  were  the  hardware  parameters  we  chose  to  minimize  such  that  the 
perceptual  fidelity  of  the  haptic  sensations  would  not  be  significantly  degraded.  Testing 
revealed  that  an  effective  stylus-based  haptic  interface  does  not  need  to  provide  the  user 
with  an  effective  bandwidth  higher  than  50  Hz  and  does  not  need  to  produce  a  maximum 
force  output  beyond  2.5  lb.  Other  experiments  also  showed  that  increasing  the  apparent 
weight  or  inertia  of  the  structure  significantly  degrades  manual  dexterity. 

Appendix  C  expands  upon  this  data  by  exploring  the  responses  generated  by  users  of  a 
single  degree-of-ffeedom  force  reflecting  device  like  the  one  used  in  the  epidural  analgesia 
simulator  developed  for  this  project. 


Evaluation  of  Perceptual  vs.  Physical  models  for  Haptic  representations 

Appendix  C  contains  a  paper  which  examines  the  modeling  of  haptic  percepts,  comparing 
the  use  of  perceptual  and  physical  models. 

Physical  modeling  involves  developing  a  mathematical  representation  based  on  the 
physics  of  the  procedure  itself.  Physical  modeling  for  an  epidural  simulator  involves  the 
development  of  a  mathematical  representation  of  the  needle/tissue  mechanical  interaction 
with  regard  to  reaction  forces.  Such  a  physical  model  would  accoimt  for  pertinent 
physical  properties  of  the  tissue  and  even  tissue  layering.  Using  this  model,  the  simulator 
system  would  derive  the  reaction  force  as  a  function  of  needle  insertion  depth  or  insertion 
velocity  and  would  reflect  that  force  to  the  user.  In  essence,  a  physical  model  would 
represent  both  the  biological  tissue  and  needle  and  predict  the  behavior  of  their 
interaction. 

While  it  is  reasonable  to  assume  that  a  simulator  which  can  represent  the  exact  physical 
behavior  of  the  needle/tissue  interaction  will  be  perceived  by  a  user  as  feeling  exactly  like 
an  epidural  needle  insertion,  it  is  not  necessarily  the  case  that  a  simulator  which  almost 
behaves  like  a  needle/tissue  interaction  will  be  perceived  as  feeling  almost  real.  The 
information  which  is  missing  or  distorted  may  be  insignificant  from  a  physical  modeling 
perspective,  but  may  in  fact  have  contained  the  salient  perceptual  cues  on  which  a  user 
was  relying  [Rosenberg  1993].  Since  hardware  limitations  prevent  even  the  state-of-the- 
art  force  reflecting  systems  from  perfectly  representing  physical  interactions,  it  is 
inevitable  that  the  physical  model  will  not  be  complete.  Since  physical  modeling  provides 
no  indication  as  to  which  information  is  perceptually  important  to  the  user,  physical 
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models  have  the  potential  to  be  inefficient  or  incomplete  virtual  representations.  For 
example,  when  generating  a  haptic  simulation  of  a  rigid  surface  with  a  physical  modeling 
approach,  one  might  strive  to  produce  an  infinite  stiffness  spring.  Since  force  feedback 
hardware  cannot  produce  a  spring  of  infinite  stiffness,  the  approach  may  be  to  produce  as 
high  a  stiffness  as  hardware  will  allow. 

A  perceptual  analysis  of  a  rigid  surface  has  revealed  through  human  testing  that  the 
perception  of  encountering  a  rigid  surface  is  not  as  strongly  correlated  to  stiffness  as  it 
correlated  to  the  intensity  of  the  initial  jolt  [Rosenberg  Adelstein  1993,  Rosenberg  1994]. 
The  indication  being  that  an  adequate  perceptual  model  may  be  very  different  from  an 
accurate  physical  model  depending  upon  the  relevant  perceptual  cues.  Thus  we  can 
define  a  perceptual  model  of  a  virtual  haptic  sensation  as  one  which  reproduces  the  salient 
perceptual  properties  (i.e.  the  feel  of  the  interaction)  rather  than  representing  the  salient 
physical  properties  (i.e.  the  behavior  of  the  interaction).  Such  an  approach  generally 
requires  human  testing  to  derive  the  required  perceptual  features  but  it  has  the  potential 
of  resulting  in  a  more  efficient  and  effective  virtual  representation. 


Figure  C-6:  CT-Scan  Data 
Physical  and  Perceptual  Model  Development: 

The  model  represents  the  virtual  tissues;  it  is  responsible  for  managing  the  reaction  forces 
generated  when  the  needle  passes  through  the  various  layers  of  virtual  tissue  during  the 
simulated  epidural  procedure.  Two  pieces  of  data  closely  associated  with  the  epidural 
analgesia  procedure  and  which  are  used  for  training  purposes  can  provide  starting  points 
for  the  development  of  good  models.  One  of  these  pieces  naturally  inspires  a  physical 
model  and  the  other  a  perceptual  model.  In  the  following,  we  shall  introduce  both  but 
eventually  select  only  one  as  the  superior  basis  for  a  model  to  be  used  in  the  product. 

A  Preliminary  Physical  Model: 

CT-scan  or  MRJ  data  of  the  spinal  column  region  is  readily  available.  CT-scans  can  be 
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used  to  produce  a  volumetric  representation  which  roughly  corresponds  to  tissue  water 
densities.  A  linear  path  cut  through  such  a  volumetric  data  set  could  be  used  to 
interactively  report  the  water  density  of  the  virtual  tissue  at  the  tip  of  the  needle  as  it 
traverses  the  volume  under  the  trainee's  control.  Quite  conceivably,  such  data  could  be 
used  to  generate  the  reflection  forces.  The  assumption  here,  of  course,  is  that  water 
density  provides  a  good  direct  indicator  of  the  insertion  force. 

The  plot  in  Figure  C-6  shows  the  density  values  at  incremental  locations  along  the 
insertion  path  as  collected  via  CT-SCAN.  A  spline  curve  has  been  fit  to  the  sample  data. 
Tissue  layers  have  also  been  identified. 

We  have  used  this  data  in  an  interactive  lookup  table  scheme  to  implement  an  epidural 
simulator.  The  needle  position  as  it  is  read  in  from  the  encoder  is  used  as  an  index  to  the 
table  and  the  force  readout  of  the  table  is  commanded  to  the  motor.  The  field  generated 
with  this  scheme  judged  by  non  experts  did  indeed  indicate  passage  through  layers  of 
mechanical  properties.  The  property  which  varied  was  easily  identifiable  as  stiffness,  in 
particularly  because  the  force  would  persist  even  if  the  user  stopped  moving.  To  index 
the  table  not  with  position  but  rather  with  instantaneous  would  presumably  provide  a 
better  approximation  to  the  forces  reflected  from  insertion.  An  important  problem, 
though,  is  that  we  do  not  have  a  reference  or  a  means  of  judging  the  performance  of  our 
model  without  having  a  person  experienced  in  the  epidural  procedure  to  do  the  testing. 

Furthermore,  we  hypothesize  that  the  water  density  to  insertion  force  is  not  so  simple 
and  not  direct.  We  assume  that  a  tissue’s  water  density  is  a  simplistic  and  incorrect 
parameterization  of  its  mechanical  interaction  with  a  needle  tip.  Other  tissues  properties 
such  as  structure,  neighboring  tissues,  teat  strength,  and  tissue  stiffness  each  of  which  do 
not  vary  directly  with  water  density  will  certainly  play  a  role  in  determining  needle 
insertion  force.  The  physical  model  based  on  the  CT  Scan  data  was  a  natural  first  choice 
and  perhaps  a  good  starting  point,  but  we  want  to  point  out  that  the  physical  parameter 
it  reports  is  an  incomplete  picture  of  the  physics  we  want  to  find  a  substitute  for. 

A  Preliminary  Perceptual  Model: 

A  practice  currently  used  during  the  initial  training  of  anesthesiologists  provides  a 
plausible  starting  point  for  the  development  of  a  good  model.  Anesthesiologists  in  training 
start  practicing  the  procedure  by  inserting  the  epidural  needle  through  various  substitute 
materials,  usually  food  items.  The  materials  chosen  for  training  purposes  are  themselves 
data  or  sources  of  data  useful  for  the  development  of  perceptually  valid  haptic  simulation 
algorithms. 

Note  that  when  medical  instructors  teach  this  procedure  to  students  they  do  not  describe 
the  feel  of  the  epidural  insertion  in  terms  of  stiffhess  and  density  of  tissue  structures,  but 
rather  describe  the  process  through  perceptual  analogy.  For  example,  the  feel  of  the 
needle  passing  through  the  skin  is  often  described  as  puncturing  the  skin  of  a  tomato. 
Students  are  even  asked  to  practice  puncturing  the  skin  of  a  tomato  with  a  needle  to 
familiarize  themselves  with  the  haptic  percept.  Passing  the  needle  through  subcutaneous 
tissue  is  described  as  traversing  the  pulp  of  a  tomato.  Penetrating  the  Supraspinous 
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ligament  feels  like  penetrating  a  ripe  pear.  Hitting  the  bone  is  like  sticking  into  a  cork 
board.  Although  various  instructors  use  various  perceptual  analogies,  the  teaching 
technique  is  generally  to  describe  the  haptic  landscape  in  terms  of  abstract  feel  parameters 
rather  than  concrete  physical  parameters. 

The  Table  below  lists  a  perceptual  description  of  the  feel  of  a  needle  penetrating  each 
region  of  the  insertion  path  as  provided  to  us  by  an  experienced  instructor  of  this 
procedure. 


TISSUE  REGION 

PERCEPTUAL  REPRESENTATION 

FORCE 

Skin 

Subcutaneous  Tissue 
Supraspinus  Ligament 
Interspinus  Ligament 
Ligamentum  Flavum 
Epidural  Space 

Bone 

“Feels  like  puncturing  the  skin  of  a  tomato” 

“Feels  like  traversing  the  pulp  of  a  tomato” 

“Feels  like  traversing  the  meat  of  a  ripe  pear” 

“Feels  like  traversing  the  pulp  of  a  tomato” 

“Feels  like  traversing  the  meat  of  a  ripe  pear” 

“Feels  like  you  are  exiting  a  ripe  pear” 

“Feels  like  you  hit  a  cork  board  with  a  dart.” 

Sudden  Increase 

Constant  Force 

Increase 

Decrease 

Sharp  Increase 

Sudden  Decrease 

Very  Sharp  Increase 

We  have  characterized  the  feel  of  needle  insertion  into  tomatoes  and  pears  using  the 
force  and  displacement  sensored  needle.  Figure  C-7  shows  the  force  readings  as  a 
function  of  needle  insertion  depth  for  a  layered  set  of  pear  and  tomato  slices.  The  outer 
tomato  slice  had  an  intact  skin  in  place.  The  force  profile  of  the  pear  is  visibly  different 
from  the  force  profile  of  the  tomato  as  seen  in  Figure  C-7.  The  higher  forces  in  the  first  5 
mm  of  tomato  are  due  to  the  tomato  skin. 

We  have  also  used  this  data  in  lookup  table  scheme.  In  order  to  create  the  lookup  table, 
the  date  of  Figure  C-7  was  resampled  to  a  function,  filtered,  and  scaled.  Figure  C-8  shows 
a  plot  of  the  resulting  profile  which  can  now  be  used  as  a  lookup  table. 


Figure  C-7 :  Force/displacement  Measurements  of  Real  Layered  Tomato  and  Pear  Slices 
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Figure  C-8:  Layered  Pear  and  Tomato  Profile:  Lookup  Table  for  Simulation 


Figure  C-9:  Force/displacement  Measurement  of  Virtual  Layered  Pear  and  Tomato  Slices 

Figure  C-9  shows  the  output  force  as  recorded  by  the  force  sensor  during  a  needle 
insertion  through  the  virtual  pear/tomato  slices  rendered  with  the  lookup  table  of  Figure 
C-8. 

Note  first  that  the  forces  generated  in  the  simulation.  Figure  C-9,  are  scaled  down 
significantly  from  the  forces  for  generation  of  the  lookup  table.  Figure  C-7.  This  is  due  to 
the  particular  scaling  chosen  for  the  profile,  and  is  not  necessary ,  but  does  relax 
requirements  on  the  simulator.  The  shape  of  the  simulated  force  profile  is  indeed  very 
similar  to  the  shape  for  the  lookup  table.  The  motions  made  by  the  user  during 
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simulation  were  similar  to  the  motions  made  during  the  recording  of  Figure  C-7,  that  is, 
the  same  insertion  velocities  were  used.  Had  this  not  been  the  case,  the  virtual  and  real 
force  profiles  would  have  differed  a  great  deal  more.  Such  is  the  shortcoming  of  this 
lookup  table  approach.  This  can  only  be  viewed  as  a  first-cut  simulation  attempt.  Once 
again,  the  forces  felt  were  spring  forces  rather  than  damping  forces  since  position,  not 
velocity,  was  used  to  index  the  table. 

An  Enhanced  Model 


As  noted  above,  the  use  of  table-lookup  for  the  control  law  is  not  able  to  produce  a  truly 
interactive  simulation.  A  more  complete  mechanical  model  is  needed:  one  that  represents 
the  driving-point  impedance  of  the  handle  during  needle  interaction  with  tissue.  Input  to 
such  a  model  would  include  velocity  and  perhaps  acceleration  of  the  handle  in  the  user’s 
grip  as  well  as  position.  An  impedance  characterization  of  the  needle/tissue  mechanics 
similar  to  the  kind  of  modeling  of  the  impedance  of  the  human  hand  underway  at  Harvard 
[Hagian,  1994]  would  provide  models  usable  for  fully  interactive  simulation.  A 
characterization  of  the  needle’s  insertion  through  human  tissue  will  indeed  be  part  of  this 
project.  Data  will  be  collected  from  a  cadaver  with  an  instrumented  needle  by  medical 
instructors.  But  even  before  such  models  are  available,  we  see  promise  in  perceptual 
modeling.  We  are  building  models  (control  laws)  which  on  the  surface  appear  to  be 
physical  models,  but  which  are  in  fact  perceptual  models.  The  basic  form  contains  string 
and  damping  terms.  But  instead  of  basing  the  parameter  values  on  tissue  characterization 
experiments,  we  are  selecting  them  by  trial  and  error  with  human  subjects  comparing  the 
virtual  feel  to  the  actual  feel  of  needle  insertion  into  pears  and  tomatoes.  A  Windows 
program  environment  supports  real-time  adjustment  of  the  feel-governing  parameters  by 
the  subjects.  The  experimenter  used  a  dialog  box  with  sliders  for  run-time  parameter  to 
simulate  layered  materials  of  varying  mechanical  properties.  Each  control  law  and  its 
associated  dialog  box  has  a  region  of  applicability  or  a  pertinent  range  of  insertion  depth 
which  can  also  be  edited  during  run-time.  By  asking  subjects,  we  can  come  up  with  model 
parameters  which  minimize  the  distance  between  the  feel  of  the  virtual  and  real  tissues. 
Alternatively,  the  fit  parameter  could  be  closeness  of  the  force  measurements  made  during 
interaction  with  virtual  tissue  and  sample  tissue. 

Other  aspects  of  the  insertion  feel  besides  its  springiness  and  damping  which  would  be 
worthy  of  modeling  have  been  suggested  by  the  heuristic  approach.  The  graininess  of  the 
pair,  for  example,  appears  to  be  an  extractable  perceptual  feature.  Sharpness  of  the  layer 
interfaces  is  another.  We  are  experimenting  with  various  ad-hoc  control  algorithms  aimed 
at  simulating  individual  features  which  can  be  overlaid  or  added  to  pre-existing  models. 

We  foresee  work  along  the  lines  of  that  of  Rosenberg  and  Adelstein  [Rosenberg  & 
Adelstein,  1993]  which  would  decompose  the  perception  of  the  needle/tissue  interaction 
into  a  number  of  separable  (not  necessarily  independent)  percepts. 

Finally,  the  model  can  be  further  enhanced  with  information  available  from  CT  Scan.  The 
model  can  draw  its  topology  or  geometrical  description  from  the  CT  Scan  data  yet  keep 
its  lumped  parameter  and  overlaid  feature  form  of  the  perceptual  model  described  above. 
The  variations  seen  from  patient  to  patient  which  are  available  from  CT  Scan  would  be  a 
very  valuable  addition.  The  tissue  would  each  be  identified  and  their  geometrical  form 
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maintained,  but  the  simulation  of  their  interaction  would  be  governed  by  other  models. 


Summary: 


A  number  of  models  have  been  presented,  each  of  which  might  form  the  basis  of  an 
epidural  analgesia  simulator.  Data  from  a  force  sensor  and  from  the  reports  of  users  has 
been  usefiil  for  the  generation  of  models  and  criticism  of  models.  Since  our  aim  is  to 
inspire  perceptions  in  the  users  of  the  epidural  simulator  product,  and  also  because 
adequate  physical  models  are  as  yet  unavailable,  perceptual  modeling  has  come  to  the  fore 
as  the  primary  model  development  tool. 

An  approach  which  is  suggested  by  these  results  is  one  in  which  MRI  data  is  used  to 
develop  tissue  topology,  this  data  is  matched  with  a  perceptual  model  of  the  various 
tissues,  and  force  feedback  is  presented  based  on  the  perceptual  model.  This  approach 
and  others  need  to  be  evaluated  in  further  research. 

Evaluation  of  passive  needle  insertion  simulator 

The  needle  insertion  simulator  has  been  evaluated  with  regard  to  the  amount  of  energy 
(work)  it  is  able  to  dissipate,  and  most  importantly,  with  regard  to  the  programmability 
of  that  amount  of  work.  The  needle  insertion  simulator  is  a  programmable  passive  device 
since  it  is  based  on  a  magnetic  particle  brake.  It  therefore  has  only  one  very  specific  kind 
of  mechanical  impedance  in  its  repertoire:  dissipative  laws.  Two  major  factors  in  the 
performance  of  the  brake  and  the  transmission  elements  linking  it  to  the  needle  are  of 
interest:  programmability  and  time  response.  This  write-up  addresses  only  the  first  of 
these,  programmability.  Time  response  will  be  important  to  consider  when  trying  to  use 
modulations  on  the  brake  current  to  produce  certain  haptic  effects.  Although  not 
investigated  in  this  study,  a  nominal  time  response  data  point  is  available  in  the  brake  data 
sheet. 

Programmability  is  further  broken  down  into  the  following  parameters: 

Dynamic  Range  Repeatability 

Linearity,  DAC  to  force  Inherent  Hysteresis 

Constituent  Curve:  Characterized  by  a  viscous  or  friction  or  stiction  shape? 

Preliminary  Observations: 

The  dynamic  range  is  of  course  determined  by  the  properties  of  the  brake,  under  the  range 
of  currents  provided  to  it  by  our  interface  hardware.  Dynamic  range  can  be  effectively 
reduced  if  the  mechanical  hardware  masks  the  brake-modulated  force  with  non¬ 
programmable  dissipative  effects.  The  repeatability  will  be  corrupted  by  any  position  or 
time  or  side-load  dependencies  of  the  needle/user  interaction  force.  Note  that  histories  of 
position  or  side-loads  may  also  play  a  role.  We  are  interested  in  the  relationship  between 
the  commanded  force  (in  terms  of  DAC  output)  and  the  actual  force.  Is  it  linear?  The 
shape  of  the  force  vs.  velocity  curve  will  reveal  just  what  kind  of  dissipative  element  we 
are  dealing  with.  Is  it  at  all  like  viscous  damping?  (straight  line)  Is  it  like  friction  (side 
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load  dependent-not  to  be  seen  in  f  vs.  v  plot)  or  stiction  (signum  function)?  Finally,  the 
force  vs.  displacement  curve  will  show  what  kind  of  hysteretic  effects  are  present.  Are 
they  simple  backlash  or  more  complex? 

The  present  study  investigates  the  above  questions  with  the  hysteresis  curve  (force  vs. 
displacement)  as  the  focal  point  and  primary  tool.  The  following  reasons  underlie  this 
decision:  Force  and  displacement  are  most  easily  measured  on  the  current  device.  The 
area  enclosed  in  the  hysteresis  loop  is  a  measure  of  the  work  done  (cumulative  work  done 
or  energy  dissipated  in  one  traversal  of  the  loop).  A  measure  of  repeatability  of  the 
device  can  easily  be  constructed  by  statistically  characterizing  the  work  done  in  multiple 
loop  traversals. 

Procedure 


A  force  sensor  (Harvard  design,  5-pound  Surface  Mount  Amp  with  a  gain  of  1000)  was 
mounted  onto  the  needle  insertion  simulator  in  place  of  the  needle  handle.  A  stylus 
handle  was  mounted  the  opposite  side  of  the  force  sensor  to  facilitate  force/motion  input 
by  hand.  The  12-bit  ADC  on  the  Impulse  Engine  interface  card  (see  section  C.l)  was 
used  for  force  data  acquisition.  Since  the  Interface  Card  ADC  accepts  0-10  volts  and  the 
force  sensor  puts  out  -10  to  10  volts,  a  simple  summing  amplifier  circuit  was  constructed 
on  a  protoboard  to  offset  the  force  sensor  analog  signal.  A  potentiometer  on  this  circuit 
allowed  for  interactive  centering  of  the  signal  in  the  range  of  the  ADC  and  plotting  range 
of  interest.  After  the  commencement  of  testing,  this  pot  was  not  adjusted  so  as  to  not 
require  recalibration.  The  Immersion  postal  scale  (accurate  only  to  5  grams)  and  a  set  of 
fishing  weights  were  used  to  calibrate  the  force  sensor.  The  serial-port-attached  “HCI- 
3.0”  Board  was  used  to  acquisition  the  encoder  counts.  The  stylus  handle  was  manually 
moved  back  and  forth  while  the  force  and  displacements  were  reeorded  at  approximately 
100  Hz  with  the  use  of  a  Windows  program  (docvw3d.ide).  Real-time  force  or  encoder 
vs.  time  and  force  vs.  encoder  reading  plots  were  available  during  testing  from  the 
Windows  application.  Manual  motions  were  made  at  approximately  1  Hz  through  a  half¬ 
stroke  (0.5  in)  placed  approximately  in  the  center  of  the  displacement  range  of  the 
simulator.  Hitting  the  endstops  during  test  was  avoided.  TTie  DAC  output  was  set  from 
the  Windows  interface  at  either  0  (low),  2048  (medium),  or  4095  (high)  and  held  constant 
during  an  entire  test.  Two  additional  tests  reported  at  the  end  of  this  document  (linearity 
and  speed  dependence)  have  procedures  which  deviate  from  the  above  in  the  following 
manners:  The  DAC  output  was  set  from  0  to  4000  in  increments  of  100  for  the  linearity 
test.  The  manual  motion  varied  from  approximately  .5  Hz  to  approximately  8  Hz  (1 
cycle  equals  one  back  and  forth  motion)  for  the  speed  dependence  test. 

DATA  ANALYSIS  and  RESULTS 


All  collected  data  was  calibrated  into  units  of  meters  and  Newtons  with  the  following 
straightforward  conversions.  The  throw  of  the  needle  was  1.04  inches  with  792  encoder 
counts.  See  Figure  C-10  for  the  calibration  data  from  force  reading  to  Newtons.  Two  sets 
of  calibration  data  were  collected  approximately  2  hours  apart.  Both  sets  are  plotted  in 
Figure  C-10.  Very  little  drift  is  apparent  from  this  plot.  Note  that  calibration  data  was 
collected  for  compressive  forces  only  (as  was  convenient)  but  was  used  to  calibrate  both 
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compressive  and  tensile  force  readings. 


Force  Sensor  Calibration  Data 


Figure  C-10:  Force  Calibration  Data 

Two  designs  were  characterized:  Device  1-The  bare  aluminum  prototype  (with  poor 
bushings  but  low  backlash)  and  Device  2-  the  anodized  second  prototype  (with  better 
bushings  but  bad  backlash). 

Typical  force  vs.  displacement  plots  for  Device  1  are  shown  in  Figure  C-1 1.  Four  loop 
traversals  (back  and  forth  motions)  each  were  made  with  a  DAC  =  0  setting  and  with  a 
DAC  =  4095  setting.  Note  that  a  larger  range  of  motion  was  executed  for  the  DAC  =  0 
reading  (no  reason).  Figure  C-12.  Shows  the  same  plots  for  Device  2.  In  comparing 
Figures  C-1 1  and  C-12,  one  may  note  that  Device  2  (the  newer  design)  has  less  dynamic 
range  than  Device  1 .  In  particular,  the  DAC  =  0  curves  have  a  larger  vertical  dimension. 
Unfortunately  only  three  full  loop  traversals  were  executed  for  Device  1  whereas  eight 
were  executed  for  Device  2. 


Device  I;  Hysteresis.  inner->0AC>0,  outef->DAC=40S5 


Figure  C-1 1: 

Force  v^.  Displacement  Plots 
for  Device  1:  Prototype  #7 


Device  II:  Hysteresis.  innor->DAC«0.  outef->DAC»4095 


Figure  C-12: 

Force  V5.  Displacement  Plots 
for  Device  2:  Prototype  #2 
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In  order  to  extract  some  repeatability  measures  from  the  above  data,  the  following  data 
analysis  was  completed.  The  zero  crossings  of  the  force  trace  were  found  with  a 
MATLAB  function  'crossup.m'.  See  Figure  C-13  for  an  example.  These  were  then  used 
as  start  and  stop  indices  into  both  the  force  and  displacement  traces  to  chop  the  plots  of 
Figures  C-1 1  and  C-12  into  separate  loops.  Once  single  loop  traversals  were  in  hand,  the 
area  enclosed  by  a  loop  could  be  found.  That  area  equals  the  work  done,  in  units  of 
Newton-meters.  The  area  was  found  with  a  simple  trapezoidal  integration  performed  on 
the  force  vs.  displacement  data.  Figure  C-14  shows  a  typical  single  loop  after  extraction. 
To  allay  the  dependence  each  of  the  area  data  thus  extracted,  each  area  was  normalized 
with  the  range  of  motion  used  for  that  particular  loop.  The  data  reported  below  are 
therefore  in  units  of  Work  per  meter  travel. 


Force  history,  showing  use  Of  CROSSUP.M  to  chop  loops 


Zero  Crossings  of  the  Force  Trace 


loop  extracted  with  the  help  of  CROSSUP.M 


Figure  C-14: 

Typical  Single  Loop  After  Extraction 


The  sample  size  (n),  mean  (x-bar)  and  standard  deviation  (sigma)  for  Device  1  and  Device 
2  taken  at  each  of  three  DAC  output  settings  (0-low,  2048-medium,  and  4095-high)  are  as 
follows: 


Device  1 
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iorca,  N  &  encoder  &  velocity 


Note  that  Device  1  does  indeed  seem  to  have  a  larger  dynamic  range,  but  Device  2  has 
better  repeatability  (though  the  sample  size  differences  do  not  allow  conclusions  about 
repeatability  to  be  drawn  with  any  confidence). 

Figure  C-15  shows  data  used  to  construct  a  plot  of  the  Constituent  Law  for  the  needle 
insertion  simulator.  A  range  of  velocities  was  produced  manually  by  varying  the  input 
frequency  of  the  back  and  forth  motion  from  about  .5  Hz  to  8  Hz,  as  seen  in  this  plot. 

The  velocity  signal  (top  trace)  was  derived  from  the  encoder  signal  by  first  differencing, 
then  filtering.  The  filtering  was  done  with  a  Butterworth  filter  (butter.m)  set  with  a 
cutoff  frequency  equal  to  .6  times  the  Nyquist  Frequency  (50  Hz,)  that  is,  30  Hz.  The 
force  vs.  velocity  trace,  or  Constituent  Law,  is  shown  in  Figure  C-16.  It  is  not  a  line. 
Viscous  damping  does  not  describe  the  brake,  as  expected.  Most  striking  in  this  plot, 
however  are  the  loops  appearing  at  high  velocities.  These  are  possibly  due  to  backlash  in 
the  coupling  fi'om  brake  to  needle  handle,  though  more  testing  would  be  needed  to  verify 
this  fact.  Velocity  is  of  course  of  highest  magnitude  when  traversing  the  middle  of  the 
stroke,  not  the  ends.  It  would  seem  that  there  is  a  dependence  on  the  change  in  velocity, 
that  is  whether  it  is  slowing  down  or  speeding  up,  on  the  part  of  the  force.  That  is,  a 
dependence  on  the  acceleration!  This  is  certainly  plausible.  This  plot  might  be  showing 
the  effect  of  the  finite  mass  of  the  moving  parts  of  the  device  (including  the  sprung  half  of 
the  force  sensor).  At  this  time,  a  full  investigation  into  the  causes  of  this  loopy  shape 
have  not  been  undertaken. 


Figure  C-15:  Figure  C-16: 

Constituent  Law  Data  Constituent  Law 


Finally,  the  linearity  of  the  device  (DAC  output  to  force)  was  checked.  The  DAC 
output  was  incremented  in  steps  of  100  while  a  consistent  back  and  force  motion  was 
input  at  the  needle  handle.  The  force  data  vs.  index  is  shown  in  Figure  C-17.  The  timing 
of  the  test  was  such  that  the  index  can  approximately  be  read  as  100  times  the  DAC 
output.  This  data  is  not  consistent  enough  to  draw  any  more  than  rough  conclusions 
fi-om .  A  more  regular  motion  input  is  needed  which  would  require  a  motor  under  motion 
(or  force)  control  to  produce  instead  of  a  human  tester. 
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DISCUSSION 


Strictly  preliminary  conclusions  can  be  drawn  based  on  this  data.  Repeatability  and 
dynamic  range  can  be  improved  upon  by  device  design  improvements. 

Further  testing  on  the  time  response  of  the  device  will  be  undertaken. 

Finally,  in  order  to  produce  cleaner  data  and  allow  more  results  to  be  drawn  from  the  data 
collected,  this  experiment  will  be  performed  again  using  a  motor  to  actuate  the  needle. 
Knowing  the  mass  of  the  moving  parts  and  the  sprung  part  of  the  force  sensor  will  allow 
extraction  of  the  acceleration  dependency  of  the  force  vs.  velocity  curve. 

Quantification  of  force  feedback  requirements  from  OSC  data 

The  team  at  OSC  has  recorded  a  set  of  data  from  an  epidural  procedure  which  does  indeed 
extract  and  display,  in  quantitative  and  qualitative  detail,  a  number  of  haptic  cues  which 
arise  from  the  interaction  of  an  epidural  needle  with  a  tissue  specimen.  The  data  set  is 
most  notable  for  its  breadth,  that  is,  its  use  of  multimedia.  A  time-stamped  video 
recording  (with  audio  track,)  was  made  simultaneously  with  a  computer-mediated 
recording  of  force,  6-D  position  and  again,  time.  The  correspondence  of  certain  features 
in  the  computer  data-file  with  certain  haptic  cues  can  be  reliably  established  by 
synchronizing  the  two  recordings. 

Certain  haptic  cues  which  the  surgeon  noted  and  verbally  identified  are  almost  visually 
discernible  on  the  video  recording.  They  appear  as  quick  motions  of  the  needle  (evidently 
through  harder-to-pierce  tissue  layers)  following  brief  periods  of  little  or  no  motion. 

More  valuable  for  the  design  of  synthetic  haptic  cues,  however,  will  be  the  computer- 
recorded  force/position  data,  so  long  as  features  in  that  data  can  be  clearly  extracted  and 
identified  as  the  haptic  aspects  of  certain  events  in  the  surgical  procedure. 

Although  still  rmder  construction,  an  overlay  of  certain  animated  plots  from  the  computer 
recordings  upon  the  video  recording  are  most  informative.  Overlays  of  the  needle- 
insertion  depth  versus  force  plot  and  the  time  versus  force  plots,  for  example,  make 
evident  which  features  of  these  plots  correspond  to  which  events  of  the  procedure.  The 
axes  of  the  plot  are  laid  out  such  that  the  full  range  in  both  position  and  force  to  be 
traversed  by  the  plot  fit  on  the  video  screen.  The  position-force  coordinate  pair  is 
graphed  on  the  plot  in  time-synchrony  with  the  video.  That  is,  a  point  shows  up  at  the 
time  of  recording  as  a  large  bold  circle,  at  its  proper  coordinate  position,  and  moves 
leaving  behind  a  trail.  By  the  end  of  the  sequence,  the  full  plot  is  drawn  on  the  screen. 

These  animated  plots  are  essentially  a  scientific  visualization  of  what  the  surgeon  felt 
during  the  procedure.  For  example,  since  forces  on  the  needle  are  generally  compressive 
during  insertion  and  tensile  during  extraction,  a  trace  on  a  plot  corresponding  to  a  needle 
insertion-extraction  sequence  traverses  a  counter-clockwise  box-like  shape.  The  direction 
of  traversal  is  counter-clockwise  in  the  present  case  since  insertion  is  graphed  positive  to 
the  left  and  compression  is  positive  toward  the  top.  The  first  video  sequence  which 
records  four  needle  insertion-extraction  sequences  (the  final  ending  in  a  puncture  into  the 
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epidural  space,)  is  in  accordance  with  an  animated  traversal  of  four  boxes  in  the  position- 
force  plot.  Finer  features  of  the  plot  (our  real  interest)  quite  obviously  correspond  to 
force/position  events  instigated  by  the  surgeon.  For  example,  extra  small  loops  at  the  far 
left  of  the  box-shape  correspond  to  probing  motions  against  the  relatively  hard  lamina. 
Undulations  in  the  force/position  graph  can  be  seen  to  correspond  to  the  stick-slip 
behavior  of  the  needle  during  extraction. 

Unfortunately  force  calibration  data  from  these  particular  recordings  was  not  available,  so 
more  than  relative  force  measures  can  not  be  determined.  However,  the  relative  forces  of 
insertion  and  extraction  are  available,  as  are  the  force-profiles  of  finer  features  such  those 
corresponding  to  a  tissue  layer  puncture.  Fine  features  which  are  discernible  at  this  time, 
before  we  have  actually  completed  the  plot- video  overlay,  seem  to  be  between  20-50% 
the  size  in  force  amplitude  of  the  full  insertion-extraction  amplitude. 

The  data  analysis  is  still  underway  at  this  vmting,  as  is  the  production  of  a  plot-video 
overlay  to  be  sent  back  to  OSC.  The  extraction  of  the  needle  path  from  the  position  and 
orientation  data  requires  the  use  of  the  inverse-kinematics  of  the  probe  used  for  this 
experiment. 

Most  importantly,  the  execution  of  this  preliminary  experiment  has  generated  a  set  of 
recommendations  for  the  design  of  the  next  experiments.  Certain  issues  such  as  full 
position  and  force  calibration  will  be  addressed.  Special  handling  of  the  force-sensored 
needle  by  the  surgeon  so  as  to  not  generate  ancillary  features  on  the  force-position  data 
have  been  recommended. 

For  future  experiments,  we  would  like  to  try  actual  real-time  graphing  of  the  force- 
position  plot,  overlaid  on  the  video,  so  that  the  surgeon,  in  seeing  the  force-position 
features  showing  up,  can  perform  manipulations  with  the  needle  which  will  emphasize  or 
de-emphasize  these  features  while  at  the  same  time  reporting  verbally  the  haptic 
sensations. 

Evaluation  of  Active  and  Passive  Components 

In  an  effort  to  determine  the  trade-offs  between  active  and  passive  technologies  to 
identify  the  optimal  solution  for  a  force  feedback  device  to  be  used  both  in  this  project 
and  in  the  general  purpose  haptic  interfaces  which  hopefully  will  emerge  as  a  result  of  this 
effort,  we  examined  data  from  OSC,  as  well  as  their  observations  as  to  the  applicability  of 
certain  types  of  force  feedback  to  specific  surgical  procedures.  The  hardware 
implementation  at  OSC  revealed  a  number  of  these  observations,  as  discussed  in  the 
following  section. 
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C.3  Experimental  and  Development  work  at  OSC 
SIMULATION  IMPLEMENTATION  DETAILS 
Voice  activation: 


Through  voice  activation,  the  user  may  request  to  view  the  penetration  of  the  needle  and 
the  surrounding  regional  anatomy.  Through  voice  captured  files,  the  system  will  prompt 
the  user  for  which  cardinal  plane;  axial,  coronal,  or  sagittal,  is  to  be  displayed.  Once  the 
section  is  displayed,  the  probe  is  locked  so  that  the  user  may  not  move  the  needle  to  a 
new  depth.  This  limitation  has  been  imposed  so  that  the  user  does  not  become  dependent 
on  the  visual,  since  these  will  not  be  available  during  actual  practice.  The  intent  of 
presenting  the  sections  is  to  assist  them  in  building  a  cognitive  map  of  the  regional 
anatomy  and  to  correlate  haptic  sensations  with  structural  anatomy. 

Initial  Mapping  of  Force  Data: 

There  are  two  common  techniques  for  the  administration  of  an  epidural.  One  is  the 
median  approach,  by  which  the  needle  is  placed  between  the  interspaces  of  the  lumbar 
spinous  processes,  and  the  trajectory  of  the  needle  is  in  the  midsagittal  plane.  The  second 
approach  is  the  paramedian,  here  the  needle  is  inserted  a  few  centimeters  lateral  to  the 
midline.  For  expedience,  we  chose  the  median  approach  to  begin  development. 

Volume  data  of  the  regional  anatomy  of  the  lower  back  was  obtained  by  employing  a  3-D 
spoiled  gradient  echo  protocol.  This  was  performed  on  a  General  Electric  1.5  Tesla 
magnet.  To  capture  the  radio  signals,  a  custom  RF  coil  placed  directly  on  the  surface  of 
the  baek. 

To  correlate  force  data  with  magnetic  resonance  data,  a  generic  trajectory  for  a  midline 
approach  was  established  by  visually  locating  the  spinous  processes  between  Lumbar  1 
and  Lumbar  2  in  the  magnetie  resonance  data.  8  bit  intensity  values  were  extraeted  from 
volumes  that  were  traversed. 

A  digital  referent  of  the  actual  needle  was  synthesized  by  the  computer  and  its  tip  was 
tracked  in  relation  to  the  volume  data.  As  the  needle's  position  was  tracked  in  relation  to 
the  reconstruction,  the  Z  coordinate  (depth)  was  tracked  along  with  the  Z-1,  Z+1,  and 
Z+2  data  points.  We  then  splined  the  datum  referenced  by  the  Z-coordinate  of  the 
needles  tip  as  it  traversed  the  probe.  Several  splines,  including  Hermite,  Bezier,  Catmol- 
Rom,  and  Beta  were  applied  under  expert  evaluation.  Bsplines  were  foimd  to  present  the 
most  natural  feel  for  the  resistive  forces  normally  encoimtered  in  the  midline  approach. 

Currently,  the  system  is  being  modified  to  drive  haptic  forces  directly  from  the  intensity 
values  located  within  each  voxel.  We  are  exploring  various  algorithms  for  sampling 
neighboring  volumes,  e.g.,  all  immediate  volumes  surrounding  the  needle  tip  in  a  2-D  plane 
of  trajectory,  or  a  3-D  sampling  of  all  immediate  voxels  surrounding  the  needles  tip  in  a 
cubic  space. 
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The  RF  coil  produced  an  imaging  artifact  where  the  attenuation  of  the  RF  signal  skews 
the  volume  intensity  based  upon  its  distance  from  the  coil.  Therefore,  we  notice  a  higher 
signal  for  volumes  closer  to  the  location  of  the  coil,  and  a  dissipating  signal  the  further  the 
volume  from  the  RF  coil  although  both  volumes  are  from  similar  species  of  tissue,  e.g.. 
muscle,  connective  tissue.  We  are  looking  at  two  methods  to  correct  for  this 
attenuation.  h2  Once  the  MRI  data  is  corrected,  we  intend  to  normalize  the  data  sets  and 
run  them  through  a  statistical  analysis  to  identify  a  correlation  between  the  MRI  intensity 
and  the  resistance  felt  against  the  needle  at  the  voxel  located  at  the  tip  of  the  needle. 


^  John  Haselgrove  and  Manfred  Prammer:  "An  Algorithm  for  Compensation  of  Surface- 
Coil  Images  for  Sensitivity  of  the  Surface  Coil,"  Magnetic  Resonance 
Imaging, 4'A69-472(  1986). 

2  A.  Nelson,  Y.  Kim,  R.  Haralick,  P.  Anderson,  R.  Johnson,  and  L.  DeSoto:  "Stereo  and 
Multiplanar  Video  Display  of  3-D  Magnetic  Resonance  Image  Data,"  Journal  of 
Imaging  Technology,  15:74-78(1989). 

Rendering  of  the  Back: 

To  render  the  back,  we  begin  with  a  series  of  CT  slices  saved  in  RGB  format.  We  create  a 
volume  data  set  by  stacking  these  slices  on  top  of  one  another.  This  volume  data  set  is 
used  as  input  to  the  Marching  Cubes  (Lomsen,  W.E.  and  Cline)  algorithm  in  order  to 
create  an  iso-surface  made  of  small  polygons.  The  back  is  then  rendered  on  a  Silicon 
Graphics  workstations  and  the  GL  programming  library. 

The  Marching  Cubes  algorithm  creates  a  large  number  of  polygons,  which  slows  down  the 
rendering  of  the  back  to  below  real-time.  This  can  be  fixed  in  a  couple  of  ways: 

a)  Decimation  of  the  Iso-surface: 

Currently  no  decimation  algorithm  is  implement  to  reduce  the  number  of  polygons,  eg, 
Schroeser,  Zarge,  and  Lorensen  in  SIGGRAPH,  1992.  In  the  event  that  head  tracking  or 
user  viewpoint  tracking  is  required,  we  will  implement  this  algorithm  and  tests  its  efficacy 
for  delivering  real-time  performance.. 


b)  Storing  of  the  Rendered  Image: 

The  current  system  supports  re-selection  of  the  viewpoint  under  user  control  through  the 
mouse  interface.  As  the  viewpoint  does  not  need  to  be  updated  in  real-time,  we  store  an 
image  of  the  rendered  iso-surface  and  load  the  image  every  time  we  need  to  redraw  the 
scene.  Using  this  method ,  we  need  to  re-render  and  store  a  new  image  only  if  the  users 
viewpoint  has  changed.  However,  this  has  implications  when  updating  the  cast  shadow. 


c)  Modifying  the  Marching  Cubes  Program: 

Currently  we  are  testing  a  simple  way  of  reducing  the  number  of  polygons.  It  is  a  simple 
modification  of  our  marching  cubes  program.  In  this  modification  we  make  an  assumption 
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that  the  surface  we  want  is  basically  smooth,  2D,  and  takes  up  most  of  the  volume.  With 
these  assumptions  we  can  modify  marching  cubes  to  just  create  one  front  surface  which 
greatly  reduces  the  number  of  polygons.  A  further  reduction  of  polygons  could  be 
accomplished  by  running  a  decimation  algorithm  on  that  front  surface. 

Rendering  of  the  Needle: 

The  needle  is  an  iso-surface  of  polygons  created  vdth  modeling  software.  The  needles  tip 
can  be  set  any  where  parallel  to  the  back  which  allows  various  insertion  points.  The 
needles  insertion  distance  is  controlled  by  any  of  the  Immersion  devices.  However, 
visually  determining  the  depth  of  the  needle  on  a  2D  display  was  often  difficult.  Two 
methods  of  correcting  this  are . 


a)  3D  Glasses: 

Silicon  Graphics  supports  the  use  of  stereo  presentation  with  LCD  shutter  glasses, 
CrystalEyes™,  to  present  each  eye  a  different  viewpoint  of  the  objects  in  the  screen. 
This  presents  the  viewer  with  the  illusion  that  the  screen  is  in  3-D,  allowing  the  user  to 
better  determine  the  depth  of  the  needle. 


Figure  C-17:  CrystalEyes  Glasses  and  Microphone  Used 


b)  Shadows: 

Another  method  for  assisting  the  user  in  determining  visual  depth  is  the  use  of  the  cast 
shadow  of  the  needle  on  the  back.  The  cast  shadow  is  created  using  a  method 
described  by  Bergeron  and  creates  a  shadow  volume  from  the  object  that  is  casting  the 
shadow.  We  have  implemented  this  technique  through  use  of  SGI's  stencil  plane  to 
mask  out  the  pixels  in  the  shadow. 

Integration  of  Immersion  Hardware  into  Simulations 


1)  The  six  degree-of-freedom  Arm  )(lmmersion  Probe) 

When  using  the  6  degree-of-ffeedom  arm  to  interface  with  the  simulator,  the  user  is  able  to 
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move  the  graphical  representation  of  the  needle  around  and  into  the  computer-generated 
reconstruction  of  the  back  by  physically  manipulating  the  arm.  The  6  degree-of-freedom 
arm  does  not  provide  force  feedback;  the  user  receives  only  visual  cues  to  identify  the 
relative  position  of  the  needle  with  respect  to  the  back.  These  cues  include  the  cast 
shadow  of  the  needle  on  the  back  and  perspective,  giving  the  user  the  perception  of  depth 
in  the  2D  image. 

The  6  degree-of-freedom  arm  can  be  connected  to  any  SGI  workstation  via  the  RS232 
port,  specifically  a  4  processor  Power  (64  bit)  ONYX,  a  Power  Indigo2  Extreme,  a 
Crimson  VGXT,  an  Iris  Indigo,  and  the  entry  level  INDY.  Graphics  sophistication  is 
determined  by  the  level  of  graphics  hardware  available  on  each  machine. 


2)  One  degree  of  freedom  passive  force  feedback  device: 

When  using  the  1  degree-of-ffeedom  passive  probe  the  user  receives  both  visual  and 
haptic  feedback  from  the  simulator.  The  graphics  display  is  identical  to  the  display 
described  above  for  the  6  degree-of-freedom  probe. 

The  user  currently  employs  the  mouse  as  an  interface  to  select  placement  of  the  needle  on 
the  reconstructed  back.  Once  positioned,  needle  penetration  is  controlled  by  the  1  degree- 
of-freedom  probe.  Intensity  values  can  be  obtained  from  each  individual  voxel  that  is 
penetrated  by  the  tip  of  the  needle.  The  user  can  choose  to  have  current  intensity  values 
shown,  using  an  intensity  meter  displayed  in  a  small  graphics  window. 

The  passive  force  feedback  system  gives  the  user  the  feel  that  the  needle  is  being 
restricted  from  movement,  which  is  not  realistic.  For  example,  in  an  actual  epidural,  as 
the  needle  is  pressed  against  the  skin  the  anesthesiologist  feels  a  reaction  force  pushing 
back  against  the  needle.  This  active  type  of  resistance  cannot  be  modeled  using  the 
passive  force  feedback  device.  This  issue  is  addressed  by  integration  of  the  1  degree-of- 
freedom  Impulse  Engine  described  in  the  next  section. 

The  1  degree-of-freedom  passive  probe  has  been  connected  to  the  various  SGI  maehines 
via  the  RS232  serial  port  as  mentioned  above. 


3)  One  degree-of-freedom  Impulse  Engine 

The  1  degree-of-freedom  Impulse  Engine  has  not  been  connected  to  the  epidural 
simulation  system,  because  we  are  unable  to  interface  it  with  the  SGI  hardware.  This  is 
something  that  will  need  to  be  addressed  very  early  in  a  Phase  II  effort  to  allow  us  to 
utilize  the  simulation  power  of  the  SGI  workstations.  A  section  has  been  added  below  to 
comment  on  and  suggest  solutions  for  the  lack  of  SGI  interface  support  present  in  these 
Phase  I  prototypes. 

We  have  simulated  the  Pig  Study  data  and  a  standard  data  vector  from  the  MRI  data  set, 
using  the  1  degree-of-freedom  impulse  engine  connected  to  the  PC  via  the  ISA  bus.  This 
simulation  hardware  gives  more  realistic  force  feedback  than  the  1  degree-of-freedom 
passive  device  but  does  not  support  any  graphics  capabilities. 
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The  graphic  display  is  not  essential  to  the  task  of  delivering  an  epidural.  The  cognitive 
task  required  is  that  the  user  must  integrate  haptic  sensations  perceived  from  forces 
placed  upon  the  needle  with  mental  representations  of  anatomical  structures.  The 
successful  integration  of  the  sensory  map  with  a  cognitive  map  is  the  key  to  learning 
proper  epidural  technique.  So  at  this  time,  we  are  able  to  run  limited  pilot  studies  with 
experts  to  evaluate  the  quality  of  the  1  degree-of-ffeedom  Impulse  engine  for  transmitting 
lifelike  forces  encoimtered  in  epidural  administration.  As  the  cormectivity  of  the  advanced 
probe  and  more  advanced  workstations  is  solved,  we  will  integrate  the  probe  with  the 
advanced  graphics  workstation  and  continue  summative  evaluations. 


Experimental  Force  Study 

We  currently  have  a  human  subjects  protocol  to  obtain  structural  and  force  data  from 
cadaver  specimens.(Reference  same)  This  protocol  is  under  the  auspices  of  the  Ohio 
State  University  College  of  Medicine  Willed  Body  Program.  Obviously,  obtaining  a 
usable  specimen  is  not  in  direct  control  of  the  researcher. 

In  order  to  expedite  our  sampling  of  surgical  force  data,  we  decided  to  use  an  animal 
specimen  for  a  pilot  study.  Results  of  this  study  have  provided  useful  information  which 
will  be  implemented  in  the  cadaver  studies.  The  closest  anatomical  correlate  in  size  was 
the  porcine  model.  A  specimen  used  for  a  vascular  graft  study  was  obtained  from  the  Lab 
Animal  Resources  at  Tlie  Ohio  State  University  College  of  Medicine.  The  animal  had 
been  sacrificed  within  a  60-hour  period  and  was  unembalmed.  A  section  of  the  back,  from 
below  the  tenth  thoracic  vertebrae  (TIO)  superiority  to  the  ileal-sacral  joint  inferiority  and 
10  centimeters  lateral  from  the  midline  including  the  false  ribs,  was  obtained. 

The  specimen  was  fixed  in  a  wooden  box  with  surgical  tape.  No  ferrous  implements 
could  be  used  in  fixing  the  specimen  because  it  subsequently  was  placed  in  a  magnetic 
resonance  imager  to  obtain  structural  data  (see  below).  A  single  degree  of  freedom  force 
sensor  was  retrofitted  to  an  epidural  needle.  The  sensored  needle  was  fixed  to  the 
Immersion  6  degree-of-freedom  Personal  Digitizer  (see  accompanying  videotape)  using 
tie-wraps.  Spatial  coordinates  (X,Y,Z)  of  the  needle  tip  were  recorded,  along  with  the 
roll,  pitch,  and  yaw  of  the  needle,  providing  the  angle  of  the  needle  trajectory  in  relation 
to  the  anatomy.  Forces  transduced  by  the  epidural  needle  were  sampled  at  30  Hz.  The 
specimen  was  spatially  registered  with  vitamin  E  capsules,  which  were  located  by  the  6 
degree-of-freedom  digitizer  before  administration  of  the  epidural  technique.  Vitamin  E 
capsules  are  used;  as  they  are  fat  soluble,  they  are  resolved  by  the  MRI  scanner.  They 
provide  registration  marks  with  the  structural  data  from  the  MRI  and  the  force  data. 

Two  medial  approaches  and  one  paramedian  approach  were  performed.  The  expert 
vocalized  his  activities  throughout  the  procedure.  The  study  was  videotaped  on  3/4” 
Umatic  with  a  visible  time  stamp.  This  video  time  stamp  was  registered  with  the  system 
clock  on  a  486  PC,  which  was  used  to  record  the  force  data.  Wooden  splints  were 
inserted  into  the  wound  to  register  the  trajectory  of  the  technique  with  the  MRI  data. 
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Force  data  was  registered  by  linking  the  visual  time  code  with  the  force  activity.  We  then 
analyzed  the  force  data  for  significant  events  by  visualizing  the  3-D  movements  of  the 
probe  tip  and  by  graphing  the  function  of  force  vs.  time.  See  figures  C-1 8  and  C-19  for 
this  data. 


30 


Figure  C-19:  Force  vs.  Time  Data 
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Significant  areas  of  the  data  were  rechecked  by  linking  them  with  the  verbal  description 
of  die  expert  and  direct  visual  evidence  from  the  videotape.  We  then  sampled  the 
significant  forces  involved  during  penetration  of  the  ligamentum  flavum  and  entrance  into 
the  epidural  space.  This  is  the  critical  aspect  of  learning  the  epidural  procedure  - 
resistance/release.  On  the  one  hand  the  resident  must  SLOWLY  penetrate  the  resistant 
tissues  of  the  ligamentum  flavum  and  in  addition  be  prepared  to  quickly  stop  all  forward 
movement  before  penetration  into  the  subareaehnoid  spaee  that  surrounds  the  spinal  eord, 


Figure  C-20:  The  Porcine  Back  Section 
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Figure  C-21 :  MRI  Data  showing  the  needle  and  the  surrounding  tissue. 


After  the  force  study,  structural  data  was  taken  from  the  porcine  specimen  by  placing  it  a 
General  Electric  1.5  Tesla  magnetic  resonance  imager.  A  3-D  spoiled  gradient  sequence, 
with  TR  =  30  msec.  /  TE  =  4.6  msec.  /  Flip  Angle  of  40  degrees  and  a  Field  of  View  of  24 
cm,  was  employed,  providing  a  submillimeter  acquisition  with  a  resolution  of512x512x 
124.  Scan  time  was  16:24  minutes.  The  wound  trajectory  of  needle  can  be  clearly 
identified  in  the  images.) 


Expert  Trials  with  the  Device 

The  data  of  penetrating  ligamentum  flavum  was  placed  into  a  file  and  used  to  provide 
resistive  forces  to  the  1  degree-of-freedom  probe.  The  expert  who  initially  performed  the 
technique  on  the  specimen  then  provided  expert  evaluation  of  the  forces  played  back  on 
the  1  degree-of-freedom  probe.  The  reconstructed  force  sensation  was  determined  to  be 
significantly  accurate  to  warrant  further  evaluation  by  impartial  experts. 

Subsequently  3  residents  with  high  proficiency  levels  in  technique  delivery  provided 
formative  evaluation  of  the  force  reconstruction.  This  information  provided  a  more 
unbiased  evaluation,  as  the  initial  expert  was  thoroughly  familiar  with  the  study  and  the 
equipment.  Formative  evaluation  from  these  trials  concluded  that  the  reconstructed  forces 
provided  a  familiar  sensation  to  a  typical  release  as  experienced  when  breaching 
ligamentum  flavum. 
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The  limitations  were  that  with  the  active  probe,  as  the  force  sensor  used  on  the  needle 
only  sampled  forces,  we  needed  to  simulate  the  active  forces  on  the  needle.  This  is  where 
we  plan  to  do  more  tissue  modeling  of  the  elasticity  and  drag  features  of  the  penetrated 
tissues. 

Resident  Trials  with  the  Device 

On  June  26th  1995,  Three  (3)  residents  from  the  Department  of  Anesthesiology  at  The 
Ohio  State  University  College  of  Medicine  were  asked  to  assess  the  two  scenarios 
currently  in  development  for  epidural  simulation. 


PROBE  1: 

The  first  simulation  employed  the  passive-only  needle  insertion  simulator.  It  was 
performed  on  a  Silicon  Graphics  Indigo2  Power  Extreme.  Graphics  of  a  section  of  the 
back  were  presented  to  the  user.  Positioning  of  the  needle  referent  was  mouse  driven.  An 
actual  needle  was  used  in  the  haptic  probe. 

The  residents  felt  that  the  interface  was  comfortable  to  very  comfortable  to  use.  One 
resident  commented  that  the  Probe  provided  a  place  to  "rest  and  stabilize"  their  hands 
during  the  procedure. 

In  general  they  felt  the  interface  was  straightforward  to  use.  Most  of  the  comments  were 
centered  around  a  need  for  additional  visual  cues  for  orientation  (Note:  we  did  not  run  the 
version  of  the  program  that  runs  on  the  CRIMSON  VGXT  that  supports  real-time  cast 
shadows). 

Additional  comments  resulted  from  the  inability  to  angle  the  needle.  This  is  a  Phase  II 
objective. 

As  to  whether  the  sensations  on  this  device  provided  adequate  information,  the  residents 
reported  that  it  was  "too  mechanical"  and  did  not  provide  the  differences  of  tissues  or 
tissue  planes.  Consequently,  they  felt  this  did  not  provide  similar  sensations  to  those 
encountered  in  actual  practice.  A  future  objective  is  to  present  the  skin,  muscle  and  facial 
layers,  and  penetration  of  ligamentum  flavum. 

Modifications  that  were  suggested  include: 

1 .  The  ability  to  add  degrees  of  freedom  to  the  needle  penetration. 

2.  Increase  the  levels  of  resistive  forces. 

3.  Make  the  back  appear  as  real  as  possible  with  the  inclusion  of  key  landmarks  for 
orientation. 

PROBE  2: 

The  second  simulation  included  and  active/passive  resistance  probe.  It  is  a  1  degree-of- 
freedom  impulse  engine  retrofitted  with  a  epidural  needle  and  a  syringe  used  in  technique 
administration.  As  the  simulation  was  running  on  a  486  PC,  no  graphics  were  presented. 
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In  general  the  manual  interface  was  comfortable.  However,  the  residents  expressed  a 
desire  for  a  surface  to  represent  the  patients  back.  One  resident  felt  the  movements  were 
"stiff*.  All  the  residents  felt  it  was  straightforward  to  use.  One  resident  felt  the 
sensations  provided  adequate  information.  Two  did  not. 

As  to  the  sensations  being  similar,  they  felt  it  "was  very  smooth".  There  was  a 
consensus  that  the  feel  "was  there"  and  that  with  modification,  the  sensation  from  the 
probe  would  accurately  represent  the  "feel"  of  the  resistive  forces  encountered  while 
performing  the  actual  technique.  However,  currently  the  specific  release  into  the  epidural 
space  was  too  quick  and  exaggerated.  Also  the  distance  traveled  by  the  needle  was  too 
great  in  length. 

Of  key  interest  is  that  the  release  comes  from  the  syringe,  as  the  fluid  is  released.  In  this 
simulation,  the  syringe  is  fixed.  This  would  be  a  key  consideration  for  Phase  II. 

Also,  it  was  mentioned  that  it  would  be  nice  to  experience  patient  variance,  i.e.  obese 
and/or  pregnant  patients,  pediatric  models,  the  elderly,  etc.  This  is  an  objective  for 
summative  evaluations  in  Phase  II. 

Force  Feedback  Hardware  Interface  Conclusions  and  Recommendations 

Since  the  force  reflecting  hardware  to  be  developed  in  this  effort  requires  a  connection  to  a 
high-speed  graphics  workstation,  such  as  a  Silicon  Graphics  machine,  we  examined  the 
many  options  which  we  could  pursue  in  our  commercialization  efforts  for  this 
technology.  We  found  a  number  of  feasible  approaches  which  we  can  explore  further  in  a 
Phase  II  effort. 

1 )  Connect  the  force  reflecting  hardware  to  the  PC,  then  connect  the  PC  to  a  SGI  machine 
with  a  network  interface  such  as  Ethernet  or  RS232.  This  could  be  implemented  in  two 
ways: 

a)  The  PC  would  act  as  an  information  relay  machine  forwarding  position  data  from  the 
probe  to  the  SGI  and  the  calculated  force  data  from  the  SGI  to  the  PC.  No  force 
calculations  would  be  performed  on  the  PC.  Because  of  the  large  amount  of  data 
transferred,  this  implementation  would  best  be  done  with  an  Ethernet  connection. 

b)  Use  the  PC  to  calculate  the  forces.  To  do  this,  we  would  pass  position  data  through 
the  PC  to  the  SGI  (since  the  data  sets  are  too  large  to  be  stored  on  the  PC).  The  SGI 
would,  according  to  the  position,  send  a  subset  of  the  data  to  the  PC.  The  PC  would 
then  calculate  the  forces  to  be  used  with  the  probe.  Since  the  graphics  display  needs 
to  be  updated  at  roughly  60  Hz,  we  could  connect  the  PC  to  the  SGI  via  RS232.  This 
approach  would  allow  us  to  easily  integrate  the  active  feedback  probe  into  our 
current  system  after  developing  a  library  of  routines  on  the  PC  like  the  library  used 
for  the  passive  device. 

2)  Connect  the  force  feedback  hardware  to  the  SGI  Indigo2  via  its  EISA  bus.  To  do  this 
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we  would  need  a  printed  circuit  board,  as  the  board  we  currently  have  does  not  fit.  With 
a  card,  this  solution  would  be  easy  to  implement,  however  there  could  be  problems  with 
probe  update  speed  because  of  the  UNIX  operating  system.  Also,  this  is  a  single 
machine  solution—  we  would  still  be  unable  to  use  the  device  with  the  Crimson  or  the 
Onyx.  This  solution  would  be  optimal  for  the  Epidural  Simulator. 

Human  Subject  Protocol  Summary 

Two  human  subject  protocol  were  obtained  form  the  Biomedical  Sciences  Review 
Committee  for  Research  Involving  Human  Subjects  at  The  Ohio  State  University. 
94H0228,  The  Investigation  of  the  Use  and  Efficacy  of  Advanced  and  Surgical  Computer 
Interfaces  for  Biomedical  Applications  was  obtained  to  study  users  and  advanced 
interfaces. 

Available  from  FTP  site.ftp.osc.edu  under /chrm/ftp/incoming/IMM 

A  second  protocol,  94H0415,  The  Acquisition  of  Structural  and  Force  Data  from  Cadaver 
Specimens  for  Use  in  Virtual  Surgical  Simulations,  was  obtained  to  capture  high  resolution 
imaging  data,  and  surgical  force  data. 

Available  from  FTP  site,  ftp.osc.edu  under  /chrm/ftp/incoming/IMM 


D.  Publications  and  Interactions  Related  to  This  Effort 

The  results  of  several  of  the  Phase  I  activities  have  been  submitted  for  journal  publication  and  for 
presentation  at  appropriate  conferences. 

Appendix  C  contains  the  full  text  of  an  article  submitted  by  Brent  Gillespie  entitled  “Design  of 
High-Fidelity  Haptic  Display  for  One-Dimensional  Force  Reflection  Applications.”  This  paper 
was  published  in  SPIE  Vol.  2351,  Telemanipulator  and  Telepresence  Technologies  (1994). 

Appendix  D  contains  the  full  text  of  an  article  submitted  by  John  S.  McDonald,  Louis  B. 
Rosenberg,  and  Donald  Stredney  entitled  “Virtual  Reality  Applied  to  Anesthesiology.”  This 
paper  was  published  in  the  Medicine  Meets  Virtual  Reality  conference  proceedings.-  Interactive 
Technology  and  the  New  Paradigm  for  Healthcare.  K.  Morgan,  R.M.  Satava,  H.B.  Sieburg,  R. 
Mattheus,  and  J.P.  Christensen  (Eds.).  lOS  Press  and  Ohmsha,  1995. 

Appendix  B  contains  the  full  text  of  a  abstract  submission  to  the  Medicine  Meets  Virtual  Reality 
#4  conference  to  be  held  next  January.  The  submission  is  entitled  “A  Virtual  Simulation 
Environment  For  Learning  Regional  Anesthesia.” 


E.  Personnel  Associated  with  the  Research  Effort 

Dr.  Louis  B.  Rosenberg  received  Ph.D.,  MS.,  and  BS  degrees  in  Mechanical  Engineering  from 
Stanford  University.  Louis  Rosenberg  is  an  experienced  researcher  and  path-breaking  designer  of 
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immersive  human  interfaces  and  virtual  reality  systems.  He  has  professional  experience  working 
in  both  industry  and  government  laboratories,  giving  him  a  broad  understanding  of  the  user 
population.  His  academic  work  involved  studying  the  psychophysics  of  human  interaction  with 
virtual  environments.  His  doctoral  dissertation  focused  on  the  use  of  force  feedback  to  enhance 
human  performance  in  manual  tasks.  This  work  involved  testing  human  subjects  using  a  2D  force 
reflecting  joystick  at  the  NASA/Ames  Research  Center.  At  the  USAF  Armstrong  Laboratories, 
Louis  Rosenberg  developed  an  experimental  Master/Slave  system  for  telepresence  research  using 
a  full  upper  body  force  reflecting  exoskeleton  to  display  haptic  percepts.  This  involved 
extensive  subject  testing  to  study  human  performance  in  a  teleoperated  tasks  with  3-D  haptic 
display.  In  addition  to  virtual  reality  research,  Louis  Rosenberg  has  industrial  experience 
designing  both  mechanical  hardware  and  software  products  for  consumer  use.  He  also  has 
professional  experience  working  at  Hewlett  Packard  and  General  Motors  in  the  manufacture  of 
both  mass  produced  and  limited  production  parts,  giving  him  deep  insight  into  commercial 
fabrication  processes. 

Bernie  G.  Jackson  received  an  M.S.  degree  in  Electrical  Engineering  from  Stanford  University 
with  a  focus  on  adaptive  systems  theory,  audio  synthesis  and  hardware  design  applications.  He 
is  deeply  involved  in  new  digital  signal  processing  techniques  and  physical  computing  efforts. 

His  talents  are  on  the  cutting  edge  of  electrical  engineering,  and  his  ability  to  stay  current  in  the 
field  will  allow  Immersion  to  maintain  its  innovative  role  in  the  industry.  Mr.  Jackson  has 
extensive  technical  experience  in  everything  from  pure  mathematics  and  physical  science  to 
applied  circuit  design  and  software  development.  He  has  developed  a  VLSI  chip  for  sound 
synthesis  and  has  written  a  great  deal  of  software,  including  an  optical  character  recognition 
algorithm.  He  is  experienced  with  most  computer  systems  and  is  an  expert  on  computer  system 
architecture  and  interfacing  hardware. 

Richard  B.  Gillespie  holds  a  M.S.  degree  in  Mechanical  Engineering  from  Stanford  University, 
and  a  Bachelor's  Degree  in  Mecheinical  Engineering  from  the  University  of  California  at  Davis. 

Mr.  Gillespie  currently  works  part  time  for  Immersion  Corporation  while  at  the  same  time  being 
a  Ph.D.  candidate  at  Stanford  University  at  the  Center  for  Design  Research  and  the  Center  for 
Computer  Research  in  Music  and  Acoustics.  Mr.  Gillespie’s  research  activities  center  around 
the  hardware  and  controller  design  of  haptic  interfaces  for  virtual  dynamic  systems,  especially 
musical  keyboard  instruments.  He  is  currently  adapting  advanced  modeling  and  simulation  tools 
for  use  in  rendering  the  mechanical  impedance  of  a  piano  action.  User  interaction  with  complex 
virtual  mechanisms,  including  those  with  changing  kinematic  constraints  are  to  be  included  in  the 
haptic  interface/simulator  repertoire.  Mr.  Gillespie  is  also  addressing  stability  and  passivity 
concerns  with  analytical  treatments  of  these  new  controller  architectures.  Mr.  Gillespie  has 
worked  four  years  for  Hewlett  Packard  in  the  Fiber  Optics  group.  He  gained  extensive 
experience  in  the  design  and  production  of  plastic  injection-molded  and  insert-molded 
components.  Finite  Element  Analysis  and  Statistical  Process  Control  techniques  were  among  his 
developed  specialties.  His  responsibilities  included  direct  customer  interaction  and  project 
leadership.  Mr.  Gillespie  also  spent  six  months  with  IBM  designing  tooling  for  the  manufacture 
of  data  storage  devices.  He  developed  user  interfaces  and  robotic  hardware/software  for  various 
manufacturing  processes. 

Timothy  A.  Lacey  received  an  M.S.  degree  and  a  B.S.  degree  in  Mechanical  Engineering  from 
Stanford  University.  His  double  focus  included  electro-mechanical  systems  and  mechanics  of 
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materials.  Mr.  Lacey  offers  an  extensive  knowledge  of  the  use  of  materials  to  develop  safe  and 
high-quality  products.  In  addition ,  his  experience  designing  electro-mechanical  systems  and 
software  makes  him  a  vital  research  team  member.  He  is  an  experienced  researcher  and  has 
conducted  independent  work  at  the  NASA/Ames  Research  Center  (as  a  contractor  employed  by 
the  San  Jose  State  Foundation),  where  he  developed  a  new  experimental  technique  for  studying 
crack  behavior  in  welded  metal  structures  containing  significant  residual  stresses.  Mr.  Lacey  is 
also  an  associate  of  the  Center  for  Design  Research  at  Stanford  University,  where  he  is  working 
on  a  collaborative,  semi-immersive  design  environment  to  enhance  conceptual  design. 

The  Ohio  Supercomputer  Center  :  Subcontract 

Donald  Stredney  received  his  B.S.  in  Medical  Illustration  and  his  M.A.  in  Computer  Graphics 
from  Ohio  State  University.  Mr.  Stredney  is  Senior  Research  Scientist  at  the  Ohio 
Supercomputer  Center  in  Columbus,  Ohio.  His  work  involves  the  application  of  high 
performance  computing  and  advanced  interface  technology  to  the  field  of  clinical  research  and 
biomedical  education.  His  research  interest  lies  in  theories  of  representation,  specifically  the 
applications  of  computer  and  volume  graphics  to  the  representation  of  biomedical  phenomenon. 

His  current  research  includes  multimodal  studies  from  electroencephalogram  and  magnetic 
resonance  images  to  investigate  brain  mechanisms  of  addiction,  the  use  of  virtual  simulations  for 
assessing  user  performance  is  the  use  of  power  wheelchairs,  and  the  development  of  real-time 
simulators  for  use  in  surgical  pre-planning  and  resident  education.  Specifically,  these  simulations 
are  being  developed  for  endoscopic  sinus  surgery,  anterior  skull  and  cranial  based  tumor 
resection,  surgical  correction  of  congenital  heart  defects,  and  the  simulation  of  regional  anesthetic 
technique.  Mr.  Stredney  will  direct  the  development  of  the  volumetric  data  model  and  the 
graphical  interface  of  the  epidural  simulation,  and  will  serve  as  liaison  between  the  clinical 
educational  research  and  the  engineering  of  the  haptic  display. 

Dr.  John  McDonald  received  his  B.A  and  M.D.  from  the  State  University  of  Iowa.  He 
completed  residencies  in  the  Department  of  Obstetrics  and  Gynecology  ,  University  of  Iowa,  and 
in  the  Department  of  Anesthesiology  at  the  University  of  Washington,  Seattle.  Dr.  McDonald  is 
Professor  and  Chair  of  the  Department  of  Anesthesiology  and  Professor  in  the  Department  of 
Obstetrics  and  Gynecology,  at  the  Ohio  State  University  Medical  Center.  He  is  also  Director  of 
the  Department  of  Anesthesiology's  Pain  Clinic.  Dr.  McDonald  founded  the  acute  pain  service  at 
The  Ohio  State  University  Hospitals  in  1986.  He  is  known  internationally  for  his  research  in 
acute  and  chronic  pain  in  both  humans  and  animals.  His  research  includes  the  study  of  spinal 
opiod  receptors  and  anti-nociceptive  interactions  between  opiates  and  regional  anesthetic  agents. 

Dr.  McDonald  is  uniquely  qualified  as  a  domain  expert,  to  serve  as  evaluator  in  the  modeling  of 
resistive  forces,  and  to  oversee  the  overall  clinical  applications  of  the  project  as  well  as  direct  the 
resident  study. 

Consultants: 

Dr.  Bernard  D.  Adelstein  received  his  Ph.D.  from  MIT  with  a  thesis  on  the  design  of  a  high- 
performance  two-degree-of-freedom  haptic  display.  Since  that  time  he  has  been  the  chairman  or 
co-chairman  of  many  key  conferences  on  Haptic  display  and  has  published  many  papers  on  the 
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subject.  He  currently  works  for  Western  Aerospace  Laboratories  and  is  a  contracted  researcher 
for  NASA  Ames  Research  Center,  Human  Factors  and  Virtual  Reality  Lab.  His  close  proximity 
to  our  facility  will  allow  us  to  meet  with  him  and  discuss  our  designs  and  obtain  feedback. 

Bruce  M.  Schena  has  1 1  years  of  experience  in  the  design,  engineering,  and  control  of  robotic 
mechanisms  and  electromechanical  products.  He  has  a  BSME  ('86)  and  MSME  ('87)  from  MIT 
and  is  currently  pursuing  a  Degree  of  Engineer  in  Product  Design  at  Stanford  University.  He  has  a 
comprehensive  ^owledge  of  robotics,  precision  machine  design,  embedded  control,  electronics, 
firmware  development,  ergonomics,  and  production  engineering.  He  has  published  8  technical 
and  conference  papers  in  tihe  areas  of  design  and  robotics  and  has  a  number  of  patents  pending. 
His  present  thesis  work  is  entitled  “Design  and  Development  of  a  Multiple  Node  Telepresence 
System.” 


F.  Relevant  Research 

Immersion  Corporation  has  done  extensive  work  in  the  area  of  manual  interaction  with  3-D 
virtual  environments.  The  key  Immersion  persoimel  involved  on  this  project  have  had  extensive 
experience  in  the  area  of  haptic  display  for  virtual  environments.  The  following  is  a  list 
published  research  in  the  area  of  haptic  display  and  virtual  reality  which  has  been  performed  by 
the  Principal  Investigator,  other  Immersion  Corporation  Personnel,  as  well  as  the  acting 
consultants  on  this  project. 

B.  D.  Adelstein.  (1989).  A  Virtual  Environment  System  for  the  Study  of  Human  Arm  Tremor.  Ph.D.  dissertation, 
Department  of  Mechanical  Engineering,  MIT,  Cambridge  MA. 

B.  D.  Adelstein.,  and  Rosen,  M.J.  (1991).  A  high  performance  two  degree-of-freedom  kinesthetic  interface.  Human- 
Machine  Interfaces  for  Teleoperators  and  Virtual  Environments,  NASA  Conference  Publication  10071, 
pp.  108-113. 

B.  D.  Adelstein.,  and  Rosen,  M.J.  (1992).  Design  and  implementation  of  a  force  reflecting  manipulator  for  manual 
control  research.  In  Advances  in  Robotics,  ed.  H.  Kazerooni,  Amer.  Soc.  Mech.  Engr,  New  York,  pp.  1- 
12. 

R.  B.  Gillespie,  Interactive  Dynamics  with  Haptic  Display.  Proceedings,  ASME  Winter  Annual  Meeting  on 
Haptic,  1993. 

R.  B.  Gillespie,  Dynamical  Modeling  of  the  Grand  Piano  Action,  International  Computer  Music  Conference 
Proceedings,  San  Jose  Ca  1992. 

L.  B.  Rosenberg,  Virtual  Fixtures  aid  Telerobotic  Control.  Proceedings  AAAI  Spring  Symposium,  Towards 
Physical  Interaction  and  Manipulation.  1994. 

L.  B.  Rosenberg  Design  of  a  Virtual  rigid  Surface:  Haptic  /  Audio  Registration.  Proceedings  CHI  94.  Boston, 
1994. 

L.  B.  Rosenberg,  The  Perceptual  Design  of  Virtual  Haptic  Sensations.  Proceedings  Virtual  Reality  Systems 
Exhibition  1993. 

L.  B.  Rosenberg,  B.  D.  Adelstein.  The  Perceptual  Decomposition  of  Virtual  Haptic  Surfaces  Proceedings  IEEE 
Research  Frontiers  in  Virtual  Reality  1993. 

L.  B.  Rosenberg.  Virtual  Fixtures:  Perceptual  Tools  for  Telerobotic  Manipulation.  Proceedings  IEEE  Virtual 
Reality  Annual  International  Symposium,  1993 
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L.  B.  Rosenberg.  The  Use  of  Virtual  Fixtures  to  Enhance  Teleoperator  Performance  in  Remote  Environments  with 
Time  Delay.  Proceedings,  ASME  Winter  Annual  Meeting  on  Haptic  Interfaces  for  Virtual  Environments 
and  Teleoperator  Systems,  New  Orleans,  Louisiana  1993 

L.  B.  Rosenberg.  The  Use  of  Virtual  Fixtures  As  Perceptual  Overlays  to  Enhance  Operator  Performance  in  Remote 
Environments.  Technical  Report  AL-TR-1992-XXX,  USAF  Armstrong  Laboratory,  Wright-Patterson  AFB 
OH,  1992.  In  publication. 

L.  B.  Rosenberg,  "The  Use  of  Virtual  Fixtures  to  Enhance  Operator  Performance  in  Remote  Environments"  SPIE 
Telemanipulator  Technology,  1993. 

L.  B.  Rosenberg.  Perceptual  Design  of  a  Virtual  Rigid  Surface  Contact .  Technical  Report  AL-TR-1993-XXX, 
USAF  Armstrong  Laboratory,  Wright-Patterson  AFB  OH,  1992.  In  publication. 


The  Ohio  Supercomputer  Center  is  involved  in  several  projects  that  are  related  to  Virtual  Reality 
and/or  the  use  of  3-D  volumetric  models.  Most  of  this  work  involves  the  volume  reconstruction 
of  CT  or  MRI  acquired  data  and  multi-modal  imaging  and  the  exploration  of  these  data  sets 
through  parallel  implementation  on  high-performance  computing. 

•  Poly  Drug  Abuse:  EEG  and  Behavior.  This  research  is  funded  by  the  National  Institute  on  Drug 
Abuse  and  is  in  collaboration  with  the  Neuropsychopharmacology  Laboratory  at  McLean 
Hospital  of  Harvard  Medical  School.  This  work  involves  the  integration  of  P300  evoked 
potentials  with  magnetic  resonance  images  to  identify  dipole  sources  in  the  human  brain.  The 
goal  of  this  research  is  to  better  understand  the  mechanisms  of  the  human  brain  during  drug 
addiction. 

•  Accessing  Diagnostic  Accuracy  of  Virtual  Simulations  of  Brain  and  Cranial  Base  Tumors.  This  is 
an  interdisciplinary  effort  to  integrate  current  surgical  practice  in  tumor  management  with  new 
and  evolving  interface  and  high  performance  computing  technology. 

•  The  Determination  of  Environmental  Accessibility  and  Wheelchair  User  Proficiency  through 
Virtual  Simulation.  This  research  is  funded  by  the  National  Institute  on  Disability  and 
Rehabilitation  Research  and  is  a  sub-project  of  the  Ohio  State  University  Rehabilitation 
Engineering  Center.  Through  computer  pre-visualization,  architectural  designs  are  examined  for 
ADA  compliance.  In  addition,  the  virtual  system  is  used  to  assess  the  user's  performance  in  using 
a  power  chair. 

•  Visualization  of  the  Heart  Through  three-dimensional  reconstruction  of  Magnetic  Resonance 
Images.  This  research  is  funded  by  Children's  Hospital  Research  Foundation,  Columbus,  Ohio. 

The  work  involves  the  in-vivo  acquisition  of  human  heart  data  for  use  in  pre-surgical  planning 
studies  that  involve  the  surgical  correction  of  congenital  heart  defects. 

In  addition  to  the  experiences  of  our  Principal  Investigator,  our  team  includes  two  consultants 
who  are  highly  experienced  in  the  fields  of  force  feedback,  dynamics,  and  human  motor  control. 
Dr.  Bernard  Adelstein  and  Brent  Gillespie  are  both  well  known  researchers  in  the  field  of  haptic 
display  and  have  produced  many  relevant  publications  including: 

Adelstein,  B.D.,  A  Virtual  Environment  System  for  the  Study  of  Human  Arm  Tremor.  Ph.D.  dissertation.  Dept,  of 
Mech.  Eng.,  MIT.,  Cambridge  MA,  1989. 

Adelstein,  B.D.,  and  Rosen,  M.J.  A  high  performance  two  degree-of-freedom  kinesthetic  interface.  "Human-Machine 
Interfaces  for  Teleoperators  and  Virtual  Environments",  NASA  Conference  Publication  10071 ,  pp.  108- 1 13, 
1991. 
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Adelstein,  B.D.,  and  Rosen,  M.J.  Design  and  implementation  of  a  force  reflecting  manipulandumfor  manual  control 
research.  Advances  in  Robotics,  ed.  H.  Kazerooni,  Amer.  Soc.  Mech.  Engr.,  New  York,  pp.  1-12,  1992. 

Gillespie,  R.B..  Dynamical  modeling  of  the  grand  piano  action.  International  Computer  Music  Conference  (ICMC) 
Proceedings,  Held:  San  Jose,  CA,  Oct.  14-18,  1992.  pp.  77-80. 

Gillespie,  R.B.,  M.  Cutkosky.  Interactive  dynamics  with  haptic  display.  Advances  in  Robotics,  Mechatronics,  and 
Haptic  Interfaces,  DSC-Vol  49,  Proceedings  of  the  1993  ASME  Winter  Annual  Meeting,  Held:  New 
Orleans,  LA,  Nov.  28-DEC  3,  1993.  pp.  65-72. 

Gillespie,  R.B..  The  virtual  piano  action:  design  and  implementation,  International  Computer  Music  Conference 
(ICMC)  Proceedings,  Held:  Aahus,  Denmark,  Oct.  14-18,  1992.  pp.  447-448. 

Gillespie,  R.B.,  L.  Rosenberg.  Design  of  high-fidelity  haptic  display  for  one-dimensional  force  reflection 

applications.  Teleoperator  and  Telepresence  Technology,  Proceedings  of  the  International  Society  of  Optical 
Engineers  (SPIE)  East  Coast  Conference,  Held:  Boston,  MA,  Oct.  15-19,  1994 

The  following  references  provide  background  information  and  related  research  on  computer  based 
medical  simulation,  visualization  techniques,  and  surgical  procedures. 

Connolly,  K.  and  Elliott,  J.  (1972)  'The  evolution  and  ontogeny  of  hand  function,'  in  N.B.  Jones  (ed.),  Ethological 
Studies  of  Child  Behavior,  London,  WEEastem  Press 

Haselgrove  et  al,  Magn.  Reson.  Imaging  4:469-472,  1986 

Kurzion,  Y.  and  R.  Yagel,  “Volume  deformation  using  Ray  Deflectors”,  submitted  to  the  6th  Eurographics 
Workshop  on  Rendering,  (appeared  also  as  OSU-CISRC-1/95-TR02). 

Rohlf,  J.,  Helman,  J.,  “IRIS  Performer:  A  High  Performance  Multiprocessing  Toolkit  for  Real-Time  3D  Graphics”, 
Proceedings  of  SIGGRAPH  ^94  (Orlando,  Florida,  July  1994),  pp.  381-395. 

Schmalbrock,  P.,  MA.Brogan,  DW.Chakeres,  VA.Hacker,  K.Ying,  BD.Clymer,  Contrast  Optimization  for 
Submillimeter  Resolution  Imaging  of  the  Inner  Ear,  JMRI  1993;3:451-459 

Schmalbrock,  P.,  J  Pruski  L  Sun,  A  Rao  JW  Monroe,  Phased  Array  RF  Coils  for  High-Resolution  Imaging  of  the 
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G.  Relationship  with  Phase  II  R  &  D 

From  Phase  I,  we  have  a  high-performance  single  degree-of-freedom  haptic  interface  employed  in 
an  effective  medical  simulation  with  force  feedback.  We  have  experimental  results  which 
quantity  the  usefulness  of  this  haptic  display  technology  in  a  real-world  medical  training 
application.  We  also  have  a  strong  technical  foundation  to  apply  to  the  full  3-D  haptic  interface 
system  in  Phase  II. 

Phase  II  objectives  will  consist  of  developing  a  complete  working  prototype  of  a  six-degree-of - 
freedom  stylus-based  haptic  display  interface  system.  This  will  require  designing  and  building 
the  mechanism  to  size,  weight,  inertia,  and  resistance  specifications  determined  in  Phase  I.  This 
will  also  include  incorporating  sensor  and  force  feedback  hardware  that  meets  the  bandwidth, 
resolution,  and  resistance  specifications  derived  in  Phase  I.  The  Phase  II  effort  will  also  involve 
refining  and  completing  the  control  electronics,  interface  electronics,  and  software  drivers  to 
complete  the  3-D  system.  The  final  part  of  Phase  II  will  involve  developing  sample  software 
applications  which  demonstrate  the  merits  of  the  haptic  interface  system.  Ohio  Supercomputer 
Center  will  extend  the  medical  simulation  system  developed  in  Phase  I  into  3-D  and  take 
advantage  of  the  full  six-degree-of-freedom  haptic  display  developed  in  Phase  II.  At  the  end  of 
the  Phase  II  contract  period,  we  hope  to  have  a  commercially  viable  haptic  simulation  platform 
for  3-D  medical  applications.  This  would  have  direct  implications  to  general  surgical  and 
microsurgical  techniques,  including  telesurgery  or  telerobotics.  Future  research  would  include 
model  deformation  to  the  visuals.  This  would  provide  additional  perceptual  congruency  to  the 
resistive  forces  being  reflected  to  the  user. 


H.  Commercial  Applications  Potential  and  Technical  Feasibility 

H.l  Wide  Range  of  Interdisciplinary  Potential 

Commercial  potential  for  a  force-reflecting  user  interface  is  very  broad-based.  Given  that  most 
commercial  user-computer  interfaces  employ  a  "point-and-click"  approach  with  a  standard 
"mouse,"  it  is  clear  that  physical  manipulations  are  becoming  more  and  more  a  part  of  practical 
computing  in  all  professional  fields.  No  matter  what  an  on-screen  image  represents,  the  ability  to 
manipulate  it  and  feel  it  simultaneously  can  only  add  to  the  user's  power  to  quickly  perceive  and 
control  what  the  computer  is  doing.  The  relatively  low  cost  of  the  proposed  approach  to  force 
reflection  makes  commercialization  feasible  in  many  different  fields. 

Applications  range  from  desktop  file  management  to  advanced  applications  such  as  scientific 
analysis  or  computer-aided  design.  Analysts  examining  multi-dimensional  models  can  display 
information  using  haptic  characteristics  to  represent  parameters  (springiness,  inertia,  etc.). 
Without  force  feedback,  analysts  can  interactively  explore  a  data  set  only  to  the  extent  that  it  can 
be  displayed  visually,  perhaps  as  a  multi-axis  graph  or  color-coded  contour  model.  By  using 
force  feedback,  the  analyst  can  display  additional  data  that  would  have  cluttered  the  visual 
domain  or  would  have  been  otherwise  impossible  to  incorporate  in  an  intuitive  fashion. 
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H.2  Federal  Government/Military  Potential 

The  proposed  manual  interface  with  force  feedback  will  be  a  tremendously  powerful  and  portable 
tool  for  many  tasks  undertaken  by  the  federal  government.  Beyond  the  uses  mentioned  above,  a 
force-reflecting  interface  is  immediately  useful  wherever  virtual  reality  or  immersive 
environments  are  employed.  The  Haptic  Display  would  be  ideal  for  making  virtual  training 
procedures  more  realistic  by  adding  physical  feel  to  the  sensory  domain.  The  device  would  be 
important  for  navigation  through  computer-generated  landscapes  or  maps:  the  implementation  of 
force  feedback  could  allow  navigation  and  surveillance  to  be  performed  by  feel  as  well  as  by  sight. 
The  device  could  also  be  used  for  telemanipulation,  where  force  feedback  has  been  shown  to 
enhance  operator  performance.  For  example,  haptic  feedback  could  be  used  to  enhance 
performance  in  remote  repairs  or  in  the  telerobotic  cleanups  of  hazardous  materials. 

H.3  Anticipated  Benefits  and  Immediate  Commercial  Niche 

Presently,  there  are  no  commercial  products  for  haptic  display  on  the  consumer  market.  A  few 
small  firms  do  manufacture  very  expensive  force  reflecting  interfaces  to  serve  the  research 
community,  but  those  products  are  not  cost  effective  for  any  real  world  applications.  By 
implementing  the  innovative  hybrid  haptic  display  technology.  Immersion  Corporation  expects 
to  produce  a  commercial  haptic  interface  which  is  not  only  cheaper  to  produce  than  traditional 
mechanisms  for  force  feedback,  but  is  also  inherently  safer  than  typical  force  reflection 
technologies,  due  to  its  passive  components.  As  a  result,  end  users  have  been  imable  to  take 
advantage  of  force  reflection  in  real  world  applications. 

Our  goal  is  to  develop  a  commercial  product  which  is  in  a  comparable  price  range  with  other 
computer  peripherals  such  as  monitors,  printers,  and  data  storage  devices.  Such  a  product  would 
empower  computer  users  to  interact  with  computational  information  in  the  same  way  that  they 
interact  with  information  in  the  real  world,  using  their  natural  perceptual  facilities  to  analyze  and 
interpret  the  display. 

Of  the  many  potential  applications  of  Force  Feedback  technologies.  Immersion  Corporation 
plans  to  focus  on  two  primary  markets:  Engineering  and  Medical  Simulation.  These  are  markets 
which  Immersion  Corporation  currently  targets  with  existing  products.  Because  of  our  current 
presence  in  these  markets,  name  recognition  and  technical  reputation  will  allow  Immersion 
Corporation  to  quickly  and  effectively  introduce  force  feedback  technologies.  The  following 
paragraphs  discuss  the  potential  benefits  which  force  feedback  can  bring  to  a  number  of 
industries. 

Medical  Virtual  Reality:  Surgical  Training 

The  initial  commercial  target  will  be  virtual  simulations  for  medical  training.  Adding  Force 
Feedback  to  Virtual  Reality  simulations  can  greatly  increase  the  physical  realism,  and  thus 
enhance  the  overall  effectiveness  of  the  application.  Within  a  virtual  environment  which  employs 
force-feedback  technologies,  users  can  interact  with  and  explore  computer  simulations  by  making 
use  of  their  natural  sensory  and  motor  skills.  Users  of  such  systems  can  obtain  an  intimate  level 
of  insight  and  understanding  of  computational  models  not  provided  by  traditional  simulation 
techniques.  Because  force  feedback  adds  physical  realism  to  simulation  environments,  force- 
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feedback  technologies  have  an  enormous  potential  in  the  training  and  planning  of  manual  tasks. 

One  the  most  exciting  applications  of  force-feedback  simulation  technologies  is  surgical 
simulation.  Computer  environments  which  realistically  replicate  the  LOOK  and  FEEL  of  medical 
procedures  could  be  used  as  teaching  tools,  allowing  manual  procedures  to  be  learned  in  computer 
labs  rather  than  using  cadavers  or  laboratory  animals.  Imagine  a  Virtual  Reality  medical 
simulation  system  so  realistic  that  users  actually  feel  the  physical  properties  of  the  tissues  and 
organs  they  interact  with.  The  simulation  could  replicate  the  feel  of  inserting  a  needle  into  human 
tissue,  the  feel  of  making  an  incision  with  a  scalpel  blade,  or  even  the  feel  of  removing  a  gall 
bladder  with  a  Laparoscopic  Instrument. 

The  idea  of  human  tissue  simulation  will  not  only  benefit  the  physician,  but  all  of  the  allied 
health  care  providers,  such  as  nurses,  physical  therapists,  and  anyone  directly  involved  with 
patient  care.  The  more  realistic  the  virtual  model  is,  the  more  likely  the  transfer  to  actual  practice. 
This  translates  into  less  patient  morbidity.  Individuals  with  more  experience  are  less  likely  to 
have  difficulty  with  performing  a  procedure.  We  all  want  the  nurse  with  many  years  of 
experience  to  start  the  intravenous  line  in  us,  not  the  student  who  may  be  performing  the 
procedure  for  the  first  time  on  a  real  patient. 

The  Virtual  Epidural  Simulator  developed  in  Phase  I  will  be  one  such  application  which  will 
serve  a  very  important  niche  in  the  field  of  medical  education.  Currently,  there  are  75 
departments  of  anesthesiology  associated  with  accredited  medical  schools,  and  an  equal  number 
of  satellite  teaching  units  are  affiliated  with  medical  centers.  At  the  present  time  there  is  no 
effective  way  to  train  doctors  on  the  epidural  procedure  without  putting  live  patients  at  risk,  and 
therefore  these  organizations  are  practically  a  captive  audience  for  a  force-feedback  system.  It  is 
foreseeable  that  a  new  method  for  teaching  regional  anesthesia  techniques  safely  and 
inexpensively  could  greatly  improve  this  aspect  of  medical  education  and  thereby  improve  the 
quality  of  health  care  delivery. 

The  system  allows  physicians  the  freedom  to  experiment  with  new  techniques  in  a  non¬ 
threatening  environment.  Also,  with  the  advent  and  proliferation  of  endoscopic  techniques  in 
other  areas  of  medicine  and  surgery,  the  basic  tools  will  be  in  place  to  simulate  heretofore 
imdeveloped  techniques.  Lastly,  the  concept  of  being  able  to  monitor  actual  physical  technique 
will  allow  for  more  adequate  testing  of  novice  surgeon  proficiency.  As  the  proposed  system  can 
be  set  up  as  a  master/slave  environment,  expert  faculty  can  “feel”  the  forces  that  a  novice  is 
utilizing,  or  conversely,  the  novice  can  “experience”  the  delicate  forces  that  an  expert  utilizes  to 
perform  a  specific  technique. 

Engineering:  CAD/CAM  and  Simulation 

Imagine  a  Computer  Aided  Design  system  which  not  only  allows  designers  to  visualize 
mechanical  parts  under  development,  but  actually  provides  realistic  force  feedback  so  that  users 
can  FEEL  the  physical  properties  of  structures  still  on  the  drawing  board.  Long  before  concept 
drawings  were  fabricated  into  physical  prototypes,  the  designer  could  feel  the  weight,  inertia, 
compliance,  even  texture  of  the  pairts  being  developed.  This  application  clearly  has  the  potential 
to  speed  the  design  process.  These  technologies  will  give  the  designer  an  intuitive  understanding 
of  the  components  under  construction  which  numbers  and  drawings  alone  can  not  provide. 
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APPENDIX  A;  Design  Specifications  Document 

In  a  force  feedback  human  interface  system,  the  flow  of  information  forms  a  closed  loop,  as 
shown  below  in  Figure  AA-1 .  The  user  can  be  seen  as  the  starting  and  ending  point  in  this  loop. 
The  cycle  begins  when  a  user  makes  a  gesture  or  physical  motion  within  the  simulated 
environment.  Sensors  track  the  user's  physical  motions  and  relay  this  information  to  the 
system's  governing  computer.  The  computer  then  uses  dynamic  algorithms  to  calculate  the 
haptic  sensations  that  the  user  should  feel  as  a  result  of  the  physical  motion  as  well  the  defined 
contents  and  structure  of  the  simulated  environment.  Haptic  sensations  are  then  produced  by 
driving  a  set  of  actuators.  The  actuators  produce  real  physical  forces  which  make  their  way  to 
the  user's  body  through  a  mechanical  transmission.  When  the  user  perceives  these  synthesized 
forces  and  reacts  to  them,  the  cycle  is  complete. 

START 
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OR  MOTION  USER  MOnON 
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COMPUTER  RECEIVES 
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FORCE  FEEDBACK 

Figure  AA-1,  Force  Feedback  System  Flow  of  Information 

The  fundamental  goal  when  designing  a  force  feedback  system  is  to  allow  the  user  to  perform 
natural  gestures  and  interactions  within  the  simulated  space  and  create  feedback  forces  that  feel 
realistic  and  convincing.  A  system  which  achieves  such  a  situation  is  described  as  having  high 
fideli^.  The  fidelity  of  a  force  feedback  system  depends  on  each  component  in  the  cycle 
described  above.  If  any  single  component  is  insufficient,  the  realism  of  the  simulated 
environment  may  be  significantly  corrupted. 

The  system's  overall  performance  is  summarized  by  its  bandwidth,  which  is  a  measure  of  the 
highest  frequency  of  motion  that  can  travel  all  the  way  through  the  system.  Haptic  sensations 
that  feel  "sharp"  or  "crisp"  require  high  bandwidth,  whereas  those  that  are  more  gradual,  or 
"mushy"  sensations,  do  not  require  high  bandwidth.  The  system's  overall  bandwidth  can  only  be 
as  good  as  the  bandwidths  of  it  sensors,  actuators,  and  mechanical  transmission  -  frequencies 
that  are  attenuated  at  any  one  component  will  not  be  able  to  travel  through  the  entire  system. 

The  fidelity  of  a  force  feedback  system  therefore  depends  on: 


a)  the  accuracy  and  precision  of  the  sensors 

b)  the  processing  speed  of  the  central  computer 

c)  the  performance  of  the  actuators 
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d)  the  transparency  of  the  mechanical  transmission. 


a)  Sensor  Performance 

A  sensor  is  primarily  characterized  by  its  resolution,  accuracy,  and  speed.  A  sensor's  resolution 
is  defined  as  the  smallest  possible  increment  between  any  two  different  measurements.  The 
sensor  resolution  needs  to  be  adequate  for  the  computer  to  detect  the  smallest  motions  that 
constitute  significant  user  activity.  For  digital  sensors,  such  as  optical  encoders,  the  resolution  is 
simply  the  difference  between  adjacent  quantized  readings.  For  analog  sensors,  such  as 
potentiometers,  the  resolution  is  more  subtle  and  depends  on  statistical  properties  of  the  output 
signal  as  well  as  the  density  of  the  eventual  quantization.  Acceptable  sensor  resolution  is  a 
parameter  governed  primarily  by  the  properties  of  the  human  perceptual  system.  With 
insufficient  resolution,  users  will  perceive  disconcerting  vibrations  and  discontinuities.  When 
designing  a  haptic  interface,  it  is  crucial  to  maintain  a  sensor  resolution  such  that  the  haptic 
signals  in  the  desired  output  frequency  range  are  well  represented. 

Accuracy  specifies  how  true  a  given  reading  is.  The  sensor  accuracy  must  be  sufficient  to  ensure 
that  a  worst-case  error  between  the  sensor  reading  and  the  true  value  does  not  cause  unacceptable 
errors  in  haptic  output,  when  propagated  through  the  system.  If  a  sensor's  inaccuracy  is  well- 
characterized,  then  it  can  often  be  compensated  for  by  the  governing  computer,  although  such 
compensations  can  become  a  computational  burden  and  slow  the  system.  For  human  in-the-loop 
systems,  absolute  sensor  accuracy  is  usually  not  as  important  a  design  parameter  as  net  sensor 
resolution.  This  is  because  the  human  perceptual  system  is  much  more  sensitive  to  relative 
changes,  or  differentials,  than  it  is  to  absolute  magnitudes. 

A  sensor's  speed  determines  how  quickly  its  output  will  respond  to  the  user's  motion.  Any  lag 
due  to  the  sensor  will  simply  add  to  the  lag  of  the  entire  system,  degrading  its  bandwidth,  and 
preventing  the  production  of  "crisp"  sensations,  such  as  contact  with  a  rigid  surface.  Thus  it  is 
desirable  to  use  sensors  with  as  little  internal  lag  as  possible.  This  often  makes  digital  sensors, 
such  as  optical  encoders,  preferable  to  analog  sensors,  such  as  potentiometers,  which  are 
typically  slower  and  require  a  time  consuming  analog  to  digital  conversion  process. 


b)  Computation  and  Communication  Latency 

Once  the  user's  motions  have  been  detected  by  the  sensors,  there  are  several  sources  of  delay.  It 
takes  time  for  the  sensors  to  transmit  their  data  to  the  governing  computer  and  for  the  computer 
to  calculate  what  output  forces  are  required  based  on  the  given  simulation.  It  also  takes  time  for 
the  computer  to  send  the  resulting  force  commands  to  the  actuators. 

While  the  processing  delay  is  a  function  of  the  simulation  complexity  and  the  processing  speed  of 
the  host  computer,  the  transmission  delay  is  a  function  of  the  communication  method  and 
protocol.  One  popular  method  for  computer  peripherals  is  RS-232,  or  serial  communication. 

This  is  one  of  the  more  widely-supported  communications  standards  across  various  computing 
platforms,  but  it  is  often  too  slow  for  haptic  display.  In  systems  with  a  small  number  of  sensors 
and  actuators,  it  may  be  possible  to  achieve  acceptably  low  latency  with  RS-232,  especially  if  a 
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clever  data-encoding  scheme  is  used.  However,  for  systems  in  which  a  high  volume  of  data  is 
flowing,  parallel  interfaces  such  as  SCSI  or  direct  motherboard  interface  may  be  necessary.  The 
system  could  also  contain  a  host  processor  and  a  fast  interface  such  as  Ethernet,  which  could 
conmiunicate  with  any  existing  computer  system  equipped  with  Ethernet. 

Our  ISA  card  solution  presented  in  Attachment  C  provides  direct  access  to  sensor  information 
for  immediate  position  input  and  direct  access  to  digital-to-analog  converters  for  immediate 
commanding  of  force  signals.  The  direct  access  is  provided  by  placing  all  the  interface 
components  onto  the  PC  bus,  removing  any  time  delays  from  communication  with  an  external 
device.  This  solution  is  not  as  attractive  from  the  compatibility  point  of  view,  but  new  bus 
standards  such  as  PCI  may  allow  us  to  produce  a  commercially  feasible  product  which  works 
with  many  existing  computing  platforms  without  customization. 


c)  Actuator  Performance 

Actuators  are  characterized  primarily  by  their  maximum  force  output  and  response  time.  The 
response  time,  as  with  the  sensors'  response  time,  will  add  to  the  system's  overall  lag  and  degrade 
system  bandwidth.  The  maximum  force  must  be  sufficient  so  that  when  fed  through  the 
mechanical  transmission  it  can  represent  the  desired  range  of  forces  for  the  given  simulation. 

Such  magnitude  requirements  are  highly  dependent  upon  the  task  at  hand. 

Requirements  for  high  actuator  force  output  can  put  significant  demands  on  the  design  of  the 
mechanical  transmission.  Typically,  an  actuator  with  high  output  capability  must  be  larger  and 
heavier  than  one  with  low  output  capability.  High-output  actuators  also  typically  require  more 
power  and  can  complicate  output  circuitry.  In  addition,  a  high-output  actuator  can  be  potentially 
dangerous  if  it  imparts  large  forces  to  a  user's  hand  or  body.  For  these  reasons,  the  haptic 
display  system  should  be  designed  with  as  small  a  maximum  force  output  as  is  acceptable  to 
represent  the  given  simulation. 


d)  Transparency  of  transmission 

The  transmission  which  lies  between  the  actuators  and  the  user  should  introduce  as  few  dynamic 
effects  as  possible  to  the  overall  system.  An  ideal  transmission  would  not  alter  the  actuators' 
output  in  any  way,  and  the  user  would  perceive  the  desired  forces  exactly.  Any  real 
transmission,  however,  is  characterized  by  stiffness,  mass,  friction,  backlash,  and  backdrivability, 
all  of  which  can  degrade  the  force  relayed  to  the  user.  Unfortunately,  it  is  nearly  impossible  to 
achieve  the  ideal  case  in  all  of  these  areas.  Instead,  any  design  must  make  trade-offs  for  the 
resulting  system  to  have  sufficient  bandwidth  for  the  required  task.  Luckily,  the  human 
perceptual  system  is  highly  forgiving  and  can  tolerate  significant  non-idealities  in  the 
transmission  without  degrading  the  perceptual  effect.  The  key  to  designing  an  effective  and 
efficient  haptic  interface  is  to  make  use  of  human  perceptual  limitations  to  reduce  the  mechanical 
design  requirements.  The  following  subsections  address  the  key  transmission  parameters. 

STIFFNESS.  It  is  desirable  that  the  transmission  between  the  actuators  and  the  user  is  highly 
rigid,  with  minimal  compliance.  Compliance  in  the  transmission  would  reduce  the  system’s 
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mechanical  bandwidth  and  limit  the  representable  frequency  content  of  haptic  information. 
Therefore,  compliant  transmission  members  such  as  belts  and  cables  should  be  avoided  or 
sized  sufficiently  stiff  for  the  dynamic  range  of  forces  to  be  produced.  If  cable  drive  systems 
are  used,  the  cables  should  be  pre-loaded  by  a  tensioning  mechanism. 

MASS.  To  maximize  the  output  efficiency  of  a  haptic  interface,  it  is  desirable  to  have  minimal 
inherent  inertia  in  the  interface  mechanism  between  the  actuators  and  the  user.  As  a  result, 
less  actuator  effort  would  be  wasted  to  accelerate  the  mechanism’s  mass,  leaving  more  force 
to  be  applied  to  the  user.  Reduced  inertia  will  also  increase  the  mechanical  bandwidth  of  the 
system  and  increase  the  representable  frequency  range. 

FRICTION  &  DAMPING.  To  maximize  the  output  force  capabilities  of  a  haptic  interface 
mechanism,  it  is  desirable  to  dissipate  as  little  energy  as  possible  to  inherent  friction  and 
damping.  Also,  for  precise  control  of  force  output,  it  is  desirable  to  minimize  non-linearities 
imposed  by  rubbing  friction  or  viscous  damping.  Minimizing  friction  and  damping  also 
helps  to  ensure  backdrivability.  Thus,  a  primary  design  consideration  for  a  3D  haptic 
interface  is  minimal  inherent  friction  and  damping.  This  implies  employing  precision 
bearing  surfaces  and  high  quality  joints  and  avoiding  using  gears  and  other  transmission 
components.  A  published  technique  to  improve  bandwidth  (Colgate,  1993)  in  active 
(motorized)  systems  is  to  add  damping  to  the  physical  system  and  then  compensate  for  it  in 
the  control  law. 

BACKLASH.  Backlash  in  the  transmission  between  actuators  and  the  user  can  corrupt  the  feel  of 
the  haptic  information  produced  at  the  interface.  Thus  backlash-prone  transmission 
components  such  as  gears,  chain  drives,  and  lead  screws  must  be  avoided  in  hardware  design. 
Backlash-free  alternatives  which  should  be  used  include  direct  drive,  cable  drive,  belt  drive, 
or  rigid  link  drive. 

BACKDRIVABILITY.  A  fundamental  requirement  of  a  force  feedback  computer  interface  is  its 
ability  to  allow  the  user's  hand  to  apply  force  and  move  the  mechanism  while  at  the  same 
time  allowing  the  mechanism  to  apply  force  and  move  the  user's  hand.  In  other  words,  the 
mechanical  transmission  which  links  the  user's  hand  to  the  actuators  must  be  completely 
backdrivable.  The  result  is  a  system  in  which  a  user  can  apply  delicate  forces  and  be  met 
with  minimal  opposition  from  the  mechanism  unless  the  computer  deliberately  induces 
resistance  forces.  Backdrivability  is  a  fundamental  design  parameter  which  excludes 
common  mechanical  transmission  elements  such  as  gears  and  lead  screws  from  consideration. 
Highly  backdrivable  transmission  alternatives  include  direct  drive,  cable  drive,  belt  drive,  or 
rigid  link  drive. 
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Overview 

The  College  of  Medicine  of  The  Ohio  State  University,  Immersion  Corporation,  and  the  Ohio 
Supercomputer  Center  have  developed  a  virtual  simulation  environment  to  teach  the  complex 
technique  of  epidural  placement.  This  methodology  was  presented  at  Medicine  Meets  Virtual 
Reality  in  January,  1995.  Since  that  time,  we  have  developed  a  working  beta  model  to  test  actual 
residents  in  anesthesiology  residency  programs  in  the  United  States.  The  system  provides  an 
intuitive  interface  for  manipulating  virtual  data  sets  that  are  three-dimensional  volume 
reconstructions  of  actual  patients.  The  model  is  flexible,  has  excellent  adaptability,  does  not  wear 
out,  and  provides  a  fresh  form  of  learning  a  technique  that,  in  the  human  model,  presents  a 
threatening  environment  and  difficult  learning  curve. 

Administering  an  epidural  block  to  a  patient  is  a  technique  that  a  resident  approaches  with  great 
improper  administration  leads  to  inadequate  pain  relief  for  the  patient  or  worse,  the  delivery  of 
the  technique  actually  causes  pain.  We  are  initiating  trials  of  the  virtual  simulation  with  residents 
who  have  previously  learned  on  the  human  model  as  a  comparison.  We  have  developed  a  method 
for  learning  that  is  not  threatening  and  that  facilitates  multiple  practice  events  without 
supervision.  Currently,  the  system  is  undergoing  evaluation  for  its  efficacy  to  raise  resident 
proficiency  levels  for  actual  human  placement  of  an  epidural  and  its  ability  to  maximize  transfer 
to  actual  technique  delivery. 

Volumetric  data  acquisition  employs  a  submillimeter  acquisition  pulse  sequence  that  provides  a 
512X512X1 24  resolution  data  set  resulting  in  a  64  Megabyte  file.  This  data  set  is  culled  for 
visible  surfaces  and  rendered  on  a  high  performance  graphics  workstation.  A  visual  referent  of 
the  needle  used  in  the  technique  is  displayed  over  the  representation  of  the  volumetric  data  of  the 
patient.  The  user  places  an  actual  needle  into  a  force  reflecting  probe.  The  position  of  the  actual 
needle  is  correlated  to  the  digital  referent,  and  force  data  is  calculated,  based  upon  the  tissues 
which  have  been  penetrated.  The  forces  are  driven  back  to  the  haptic  probe  and  presented  to  the 
user. 
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At  any  time,  the  user  may  request,  through  a  discrete  phrase  recognition  interface,  a  cardinal 
section  of  the  placement  of  the  needle  and  the  regional  anatomy.  During  graphic  presentation  of 
the  sectional  view,  the  haptic  probe  is  locked,  preventing  the  user  from  depending  on  the  visual; 
as  this  cue  would  not  be  present  during  the  actual  technique. 

We  will  present  our  methodologies  for  obtaining  force  data  during  actual  technique  delivery, 
subsequent  data  analysis,  and  resulting  use  of  information  in  force  modeling.  In  addition,  we  will 
present  our  methods  to  correlate  force  modeling  with  tissue  densities  obtained  from  the  magnetic 
resonance  data. 

We  will  present  results  from  trials  run  with  anesthesiology  residents.  These  trials  employ 
formative  evaluation  using  a  think-aloud  protocol  by  which  the  user  verbalizes  actions  and  mental 
decisions  continuously  throughout  the  trial.  System  evaluation  and  iterative  design  will  employ 
written  evaluation  tools.. 
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ABSTRACT 


This  paper  discusses  the  development  of  a  virtual  reality  platfomi  for  the  simulation  of  medical  procedures 
which  involve  needle  insertion  into  humaui  tissue.  The  paper’s  focus  will  be  the  hardware  and  software  requirements 
for  haptic  display  of  a  particular  medical  procedure  known  as  Epidural  Analgesia.  To  perform  this  delicate  manual 
procedure,  an  anesthesiologist  must  carefully  guide  a  needle  through  various  layers  of  tissue  using  only  haptic 
cues  for  guidance.  A  simplifying  aspect  for  the  simulator  design,  all  motions  and  forces  involved  in  the  task  occur 
adong  a  fixed  line  once  insertion  begins.  To  create  a  haptic  representation  of  this  procedure,  we  have  explored  both 
physical  modeling  and  perceptual  modeling  techniques.  A  preliminary  physical  model  was  built  based  on  CT-scan 
data  of  the  operative  site.  A  preliminarj'  perceptual  model  was  built  based  on  current  training  techniques  for  the 
procedure  provided  by  a  skilled  instructor.  We  compare  and  contraist  these  two  modeling  methods  and  discuss 
the  implications  of  eadi.  We  select  and  defend  the  perceptual  model  as  a  superior  approach  for  the  epidural 
analgesia  simulator. 


Introduction 


Several  research  labs  mid  small  competnies  ai-ound  the  country  ai'e  currently  involved  in  projects  aimed  at 
generating  hardware  and  softwcU’e  to  teach  medical  procedures  through  realistic  computer  simulation..^’®  Human 
interface  products  which  track  the  activities  associated  with  certain  medical  procedures  and  provide  for  interaction 
with  computerized  tissue  and  organ  models  aie  already  appearing  on  the  market.  A  Laparoscopic  Surgery 
Simulator  from  Immersion  Corp.  is  an  example  of  one  such  product.  This  tool  allows  a  user  to  wield  surgical  tool 
handles  and  perform  the  manual  manipulations  associated  with  laparoscopic  gmd  endoscopic  surgery.  Simulators 
which  can  also  provide  haptic  feedback  to  the  user  yet  whicli  meet  low  cost  requirements  are  likewise  under 
development.  Sometimes  called  force  reflecting  interfaces,  these  simulators  cJlow  for  interaction  between  user  and 


44  /  SPIE  Vol.  2351  Telemanipulator  and  Telepresence  Technologies  (1994) 

53 


0-8 1 94-1 686-X/94l$6. 00 


model  which  include  power  exchanges,  timed  and  governed  to  enhance  not  visual,  but  mechanical  realism. 
Catheter  Insertion  Simulator  from  Immersion  Corp.  falls  into  this  category.  This  devices  allows  a  user  to  insert  a 
catheter  wire  into  a  virtual  patient  while  the  computer  tracks  the  wire’s  feed  and  spin.  In  addition  to  monitormg 
manipulations,  the  computer  can  command  a  resistance  force  to  the  wire  to  simulate  the  forces  generated  when 
a  catheter  interacts  with  body  tissue.  The  result  is  a  human  interface  platform  capable  of  providing  a  reahstic 
representation  of  the  manual  task  and  whicli  can  be  used  for  training  medical  professionals. 

A  product  currently  under  development  is  the  Virtual  Epidural  Simulator  which  is  a  single  depee  of  freedom 
force  reflecting  interface  designed  specifically  to  provide  users  with  realistic  simulation  of  the  medical  procedme 
known  as  Epidural  Analgesia.  In  addition  to  the  physical  hardware  for  haptic  display,  this  project  also  reqimes  the 
development  of  software  techniques  for  generating  the  haptic  representation  of  a  n^dle  posing  through  layers 
of  tissue.  After  briefly  introducing  the  hardware  design,  this  paper  will  address  the  fabncation  of  modek  for 
simulating  the  haptic  aspects  of  the  epidural  procedure.  We  will  pay  particular  attention  to  the  need  to  balance 
engineering  and  psychophysical  considerations  in  the  design  of  these  models.  Two  approaches  to  the  process 
of  building  models  will  be  highlighted.  We  call  the  two  approaches  physical  modeling  and  perceptual  modeling. 
Running  commentary  will  be  provided  on  eacli  presented  model  as  to  whether  it  fits  into  the  physical  or  perceptual 

model  paradigm. 

We  hope  that  to  emphasize  the  modeling  process  in  the  presentation  of  these  models  will  prove  useful.  Although 
certainly  we  don’t  expect  to  be  able  to  come  up  with  directives  about  which  modeling  approach  is  appropriate  at 
any  given  time,  we  do  hope  to  spur  the  development  of  further  design  tools  by  elucidating  the  vanous  approach^ 
to  the  process  of  model  construction.  After  presenting  some  further  background,  this  paper  wiU  introduce  the 
preliminary  hardware  design,  discuss  the  generation  of  some  design  parameters,  present  a  physical  and  then  a 
perceptual  model,  and  finally  discuss  future  model  development  efforts  before  summarizing. 


Background 


Epidural  analgesia  is  one  of  the  most  frequently  used  tedmiques  for  the  prevention  of  pain  during  surgery. 
The  procedure  involves  the  injection  of  a  local  anesthetic  or  an  opiod  into  the  epidural  space  of  the  spm^  colunm. 
Once  inserted,  the  needle  does  not  change  direction,  its  motion  is  constrained  to  a  line  by  the  dense  ligaments. 
Nevertheless,  it  is  a  delicate  manual  operation.  The  anesthesiologist  must  insert  a  catheter  mto  the  epidural  space 
using  only  haptic  cues  for  guidance.  As  the  needle  passes  through  various  tissue  layers,  a  varymg  msertion  force 
is  detectable  by  fed.  By  constantly  comparing  this  feel  to  a  mental  map  of  the  anatomy  and  an  assoaated  m^ 
of  expected  fed,  it  is  possible  to  a  maneuver  the  needle  into  the  correct  space  without  damaging  the  spmal  chord. 
All  practitioners  of  the  epidural  process  must  possess  this  detailed  mental  map  of  the  n^dle/tissue  mechamcd 
interaction  with  its  associated  haptic  cues  at  the  needle  handle.  No  visual  cues  are  available  since  the  needle  s 
path  is  hidden  bdow  the  surface  of  the  skin. 

While  epidural  analgesia  is  a  common  procedure  with  wide-spread  clinical  use,  current  techniques  for  training 
students  to  perform  this  procedure  can  be  difficult  for  both  student  and  instructor.  The  problem  is  that  there 
is  no  effective  way  for  students  to  develop  the  required  mental  model  of  the  haptic  landscape  without  practicmg 
the  manual  insertion  on  actual  live  patients.  This  creates  a  dangerous  situation  for  pati^ts,  a  stressful  leammg 
environment  for  the  student,  and  a  teaching  process  which  requires  instructors  to  spend  much  time  performmg 
interactive  supervision.  What  is  needed  is  an  off-line  simulation  environment  m  which  students  c^  repeatedly 
practice  the  procedure  and  develop  the  manual  skill  without  endangering  patients  and  without  requiring  comtant 
Ltructor  supervision.  To  meet  this  need,  Immersion  Corp.,  in  collaboration  with  researchers  at  Ohio  Supw- 
comouter  Center  and  Ohio  State  University  Medical  Center,  is  working  on  a  realistic  virtual  simulation  of  the 
epidural  procedure.  This  DOD  funded  project  focuses  on  developing  haidware  ^d  software  which  can  reproduce 
the  haptic  sensations  associated  with  epidural  needle  insertion  so  realistically  that  it  can  be  used  as  an  effective 

teaching  tool. 
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systOT,  nown  as  the  Virtual  Epidural  Simulator,  would  include  a  force-reflecting  interface  which 
Tf  j  aptic  cu^  associated  with  needle  penetration  of  the  various  layers  of  biological  tissue, 

r  . ,  ®  ®  trainee  with  a  believable,  non-threatening  environment  which  encourages  the  free  exploration 

a  !"  administering  a  lumbar  epidural.  Such  interaction  and  exploration  with 

7  understanding  of  the  anatomy  and  the  intricacies  encountered  in  the 

epidural  procedme.  It  is  hypothesized  that  such  a  training  environment  will  reduce  resident  anxiety  and  limit 
the  number  of  trials  required  to  learn  to  successfully  deliver  regional  anesthesia  through  the  epidural  method 


Hardware  Description 


The  hardw^e  for  this  project  is  made  up  of  two  parts:  Haptic  Sensing  Hardware  and  Haptic  Feedback 
Hardware.  BasicaUy,  force  and  position  sensing  hardware  is  needed  to  devise  models  to  be  used  in  the  position 
sensmg  and  force  producing  Redback  hardwtu-e. 


Haptic  Sensing  Hardware 


Lx)ad  Cell 

\ 

1 

Jnear  Slide 

Needle  Handle  j 

^  Needle  Tip 

Encoder 

Haptic  Feedback  Hardware 


Figure  1:  Hardware 

Figure  1  shows  both  the  haptic  sensing  hardware  and  haptic  feedback  hardware.  The  haptic  sensing  hardware 
IS  essentiaUy  an  mstrumented  epidural  needle.  The  epidural  needle  has  been  mounted  on  an  extra  long  hght- 
weight  handle  which  is  constrained  to  linear  motion  without  twist  by  a  set  of  high  quality  bearings.  A  small 
stram  gage  outfitt^  load  cell  which  has  been  electro-discharge  machined  out  of  aluminum  comprises  the  force 
sensor.  The  amplification  circuitry  is  located  right  on  the  load  cell  on  a  surface  mount  printed  circuit  board  An 

^  “q“i«tion  of  the  force  signal  by  a 

486DX2-66  PC.  Fmally,  a  1024  count/revolution  rotary  encoder  with  a  friction  drive  wheel  on  the  needle  handle 
IS  responsible  for  transducing  the  position  of  the  needle. 


The  hardware  for  the  haptic  feedback  product  is  made  up  of  a  linear  slide  motorized  through  a  capstan 
drive  by  a  low-mertia  precision  motor.  A  highly  flexible  steel  cable  takes  three  turns  around  a  screw  puUey 
on  the  motor.  For  simplicity  and  low  cost,  we  have  decided  not  to  include  a  force  sensor  in  the  final  product 


46  / SPIE  Vo/., 2357  Teleman ipulator  and  Telepresence  Technologies  (1994) 


Therefore,  measured  force  will  not  be  available  for  use  in  a  control  law  or  table  lookup  scheme.  An  encoder  is 
again  responsible  for  tracking  position  of  the  slide.  To  minimize  inertia,  the  rod  itself  moves  and  the  platform  is 
fixed. 

The  haptic  sensing  and  feedback  hardware  is  used  in  three  configurations.  The  sensing  hardware  is  used  alone 
with  tissue  substitue  samples  to  generate  simulation  models.  The  sensing  hardware  is  then  used  together  with  the 
feedback  hardware  to  test  candidate  models.  Finally  the  feedback  hardware  alone  will  constitute  the  simulator. 

C++  code  for  Windows  on  the  PC  is  responsible  for  closing  the  control  loop  and  supporting  a  full  graphical 
user  interface  for  development  and  product  use.  Eventually,  the  system  will  include  a  graphics  workstation  for 
the  visual  presentation  of  a  virtual  needle  and  CT-scan  images. 


Design  Parameter  Generation 

Already  the  hardware  has  proven  itself  inadequte.  It  has  become  apparent  that  the  mass  of  the  force  sensor 
on  the  needle  masks  a  significant  portion  of  the  feel  of  inserting  a  needle  into  a  pear.  See  the  Perceptual  Model 
section  below  for  a  discussion  of  the  importance  of  the  pear.  Quick  experiments  with  naive  subjects  showed 
that  the  ability  to  differentiate  a  pear  from  a  tomato  was  degraded  by  the  presence  of  the  force  sensor  on  the 
needle.  Since  the  stiffness  of  the  force  sensor  is  very  high,  its  mass  must  be  culprit  in  the  masking  effect  observed. 
A  simple  lumped-parameter  mass  spring  model  of  the  needle  in  the  grip  of  a  trainee  has  been  used  to  show 
that  indeed  the  mass  of  the  force  sensor  must  be  smaller  in  the  next  design.  Figure  2  shows  some  experimental 
data  taken  to  diaracterize  the  stiffness  of  a  typical  needle  handle  grip.  The  measured  force  is  plotted  against 
measured  displacement.  The  slope  of  a  line  fit  to  the  data  estimates  the  stiffness  at  15,000  N/m.  Note  that  more 
information  such  as  damping  is  also  available  from  this  data,  but  a  simplistic  model  is  deemed  sufiicient  for  the 
present  purposes.  This  stiffness  value  used  in  two  mass-spring  models,  one  with  and  one  without  the  mass  of  the 
force  sensor  gives  two  approximate  cutoff  frequencies.  The  addition  of  the  28  gram  force  sensor  to  the  3  gram 
needle  would  then  account  for  the  lowering  of  a  cutoff  from  2200  Hz  down  to  700  Hz. 


displacement  (mm) 


Figure  2;  Grip  Stiffness  Characterization 

We  are  not  ready  to  conclude  that  vibration  frequencies  in  the  700-1000  Hz  range  could  accotmt  for  the 
masking  effect  observed,  but  tliis  ballpark  analysis  does  suggest  that  the  presence  of  the  force  sensor  could  be 
attenuating  valuable  frequency  information  within  the  range  of  human  haptic  sensitivity.  Further  experiments 
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with  an  accelerometer  are  underway  to  come  up  with  solid  design  parameters  for  a  suitable  design  of  the  haptic 
sensing  hardware. 

Certainly  the  force  feedback  hardware  introduced  above  will  not  have  a  frequency  production  capability  above 
50  Hz.  In  order  to  produce  such  frequencies  for  haptic  display,  we  are  proposing  the  use  of  a  vibrotactile  display 
such  as  that  developed  by  Howe  and  Kontaiinis.'^’^  A  small  ’proof  mass’  actuated  by  a  tiny  voice  coil  motor  could 
be  driven  with  a  waveform  containing  the  high-frequency  information. 


Physical  vs.  perceptual  modeling 


When  trying  to  replicate  the  forces  generated  when  a  needle  passes  through  the  various  layers  of  tissue  during 
an  epidural  procedure,  there  au*e  severaJ  possible  approaches  one  cam  pursue.  The  model  or  control  law  which  will 
govern  the  force  generated  for  the  user  to  feel  will  take  on  one  of  a  few  forms.  It  may  be  implemented  as  a  lookup 
table,  an  equation,  or  even  ats  a  differential  equation  within  a  numerical  integration  scheme.^  In  the  following 
sections  of  this  paper,  we  will  present  several  preliminary  models  which  we  have  used  to  date  to  re-create  the  feel 
of  the  epidural  procedure. 

Two  basic  approaches  are  available  for  generating  the  model:  physical  modeling  and  perceptual  modeling. 
Whether  a  model  is  physically  or  perceptually  based  is  somewhat  independent  of  its  form.  For  example,  an 
equation  can  be  a  physical  model  or  a  perceptual  model  depending  on  how  it  was  derived.  The  physical/perceptual 
distinction  can  also  be  used  to  judge  a  model  independent  of  its  derivation:  does  it  accurately  predict  behavior 
(physics)  or  adequately  inspire  interpretation  (percept)? 

Physical  modeling  involves  developing  a  mathematical  representation  based  on  the  physics  of  the  procedure 
itself.  Physical  modeling  for  an  epidural  simulator  involves  the  development  of  a  mathematical  representation  of 
the  needle/tissue  mechanical  interaction  with  regard  to  reaction  forces.  Such  a  physical  model  would  account  for 
pertinent  physical  properties  of  the  tissue  and  even  tissue  layering.  Using  this  model,  the  simulator  would  derive 
the  reaction  force  as  a  function  of  needle  insertion  depth  or  insertion  velocity  and  would  reflect  that  force  to  the 
user.  In  essence,  a  physical  model  would  represent  both  the  tissue  and  needle  and  predict  the  behavior  of  their 
interaction. 

While  it  is  reasonable  to  assume  that  a  simulator  which  can  represent  the  exact  physical  behavior  of  the 
needle/tissue  interaction  will  be  perceived  by  a  user  as  feeling  exactly  like  an  epidural  needle  insertion,  it  is  not 
necessarily  the  case  that  a  simulator  which  almost  behaves  like  a  needle/ tissue  interaction  will  be  perceived  as 
feeling  almost  real.  The  information  which  is  missing  or  distorted  may  be  insignificant  from  a  physical  modeling 
perspective,  but  may  in  fact  have  contained  the  salient  perceptual  cues  upon  which  a  user  was  relying.^  Since 
hardware  limitations  prevent  even  state-of-the-art  force  reflecting  systems  from  perfectly  representing  physical 
interactions,  it  is  inevitable  that  the  physical  model  will  not  be  complete.  Since  physical  modeling  provides  no 
indication  as  to  which  information  is  perceptually  important  to  the  user,  physical  models  have  the  potential 
to  be  inefficient  or  incomplete  virtual  representations.  For  example,  when  generating  a  haptic  simulation  of  a 
rigid  surface  with  a  physical  modeling  approach,  one  might  strive  to  produce  an  infinite  stiffness  spring.  Since 
force  feedback  hardware  cannot  produce  a  spring  of  infinite  stiffness,  the  approach  may  be  to  produce  as  high 
a  stiffness  as  hardware  will  allow.  A  perceptual  analysis  of  a  rigid  surface  has  revealed  through  human  testing 
that  the  perception  of  encountering  a  rigid  surface  is  not  as  strongly  correlated  to  stiffness  as  it  is  correlated  to 
the  intensity  of  the  initial  jolt.®’®  The  indication  being  that  an  adequate  perceptual  model  may  be  very  different 
from  an  adequate  physical  model  depending  upon  the  relevant  perceptual  cues.  Thus  we  can  define  a  perceptual 
model  of  a  virtual  haptic  sensation  as  one  which  reproduces  the  salient  perceptual  properties  (i.e.  the  feel  of  the 
interaction)  rather  than  representing  the  salient  physical  properties  (i.e.  the  behavior  of  the  interaction).  Such 
an  approach  generally  requires  human  testing  to  derive  the  required  perceptual  features  but  it  has  the  potential 
of  resulting  in  a  more  efficient  and  effective  virtual  representation. 
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Model  Development 


Xhe  model  represents  the  virtusl  tissues;  it  is  responsible  for  managing  the  reaction  forces  generated  when  the 
needle  passes  through  the  various  layers  of  virtual  tissue  during  the  simulated  epidural  procedure.  Two  pieces 
of  data  closely  associated  with  the  epidural  analgesia  procedure  and  which  are  used  for  training  purposes  can 
provide  starting  points  for  the  development  of  good  models.  One  of  these  pieces  naturally  inspires  a  physical 
model  and  the  other  a  perceptual  model.  In  the  following,  we  shall  introduce  both  but  eventually  select  only  one 
as  the  superior  basis  for  the  model  to  be  used  in  the  product. 


A  preliminary  physical  model 

CT-scan  or  MRI  data  of  the  spinal  column  region  is  readily  available.  CT-scans  can  be  used  to  produce  a 
volumetric  representation  which  roughly  corresponds  to  tissue  water  densities.  A  linear  path  cut  through  such  a 
volumetric  data  set  could  be  used  to  interactively  report  the  water  density  of  the  tissue  at  the  tip  of  a  virtual  needle 
as  it  traverses  the  volume  under  the  trainee’s  control.  Quite  conceivably,  such  data  could  be  used  to  generate  the 
reflection  forces.  The  assumption  here,  of  course,  is  that  water  density  provides  a  good  direct  indicator  of  the 
insertion  force. 

The  plot  in  Figure  3  shows  the  density  values  at  incremental  locations  along  the  insertion  path  as  collected 
via  CT-SCAN,  A  spline  curve  has  been  fit  to  the  sampled  data.  Tissue  layers  have  also  been  identified. 

We  have  used  this  data  in  an  interactive  lookup  table  scheme  to  implement  an  epidural  simulator.  The  needle 
position  as  it  is  read  in  from  the  encoder  is  used  as  an  index  to  the  table  and  the  force  read  out  of  the  table  is 
commanded  to  the  motor.  The  feel  generated  with  this  scheme  judged  by  nonexperts  did  indeed  indicate  passage 
through  layers  of  varying  mec2Jinical  properties.  The  property  which  varied  was  easily  identifiable  as  stiffness,  in 
particular  because  the  force  would  persist  even  if  the  user  stopped  moving.  To  index  the  table  not  with  position 
but  rather  with  instantaneous  velocity  would  presumably  provide  a  better  approximation  to  the  forces  reflected 
from  insertion.  An  important  problem,  though,  is  that  we  do  not  have  a  reference  or  means  of  judging  the 
performance  of  our  model  without  a  having  person  experienced  in  the  epidural  procedure  to  do  the  testing. 
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Furthermore,  we  hypothesize  that  the  water  density  to  insertion  force  relationship  is  not  so  simple  and  not 
direct.  We  assume  that  a  tissue’s  water  density  is  a  simplistic  and  incorrect  parameterization  of  its  mechanical 
interaction  with  a  needle  tip.  Other  tissue  properties  such  as  structure,  neighboring  tissues,  tear  strength,  and 
tissue  stiffness  each  of  which  do  not  vary  directly  with  water  density  will  certainly  play  a  role  in  determining 
needle  insertion  force.  The  physical  model  based  on  the  CT-scan  data  was  a  natural  first  choice  and  perhaps  a 
good  starting  point,  but  we  want  to  point  out  that  the  physical  parameter  it  reports  is  an  incomplete  picture  of 
the  physics  we  want  to  find  a  substitute  for. 


A  preliminary  perceptual  model 


A  practice  currently  used  during  the  initial  training  of  anesthesiologists  provides  a  plausible  starting  point 
for  the  development  of  a  good  model.  Anesthesiologists  in  training  start  practicing  the  procedure  by  inserting 
the  epidural  needle  through  various  substitute  materials,  usually  food  items.  The  materials  chosen  for  training 
purposes  are  themselves  data  or  sources  of  data  useful  for  the  development  of  perceptually  valid  haptic  simulation 
algorithms. 


Note  that  when  medical  instructors  teach  this  procedure  to  students  they  do  not  describe  the  feel  of  the 
epidural  insertion  in  terms  of  stiffness  cind  density  of  tissue  structures,  but  rather  describe  the  process  through 
perceptual  analogy.  For  example,  the  feel  of  the  needle  passing  through  the  skin  is  often  described  as  puncturing 
the  skin  of  a  tomato.  Students  are  even  asked  to  practice  puncturing  the  skin  of  a  tomato  with  a  needle  to 
famiharize  themselves  with  the  haptic  percept.  Passing  the  needle  through  subcutaneous  tissue  is  described  as 
traversing  the  pulp  of  a  tomato.  Penetrating  the  supraspinous  ligament  feels  like  penetrating  a  ripe  pear.  Hitting 
the  bone  is  like  sticking  into  a  cork  bo2u*d.  Although  various  instructors  use  various  perceptual  analogies,  the 
teaching  technique  is  generally  to  describe  the  haptic  landscape  in  terms  of  abstract  feel  parameters  rather  than 
concrete  physical  parameters. 


The  Table  below  lists  a  perceptual  description  of  the  feel  of  a  needle  penetrating  each  region  of  the  insertion 
path  as  provided  to  us  by  an  experienced  instructor  of  this  procedure. 


Insertion  through: 

skin 

subcutaneous 
supraspinous  ligamet 
interspinous  ligament 
ligamentum  flavum 
epidural  space 
bone 


Feels  like: 

puncturing  the  skin  of  a  tomato 
traversing  the  pulp  of  a  tomato 
traversing  meat  of  ripe  pear 
traversing  pulp  of  tomato 
traversing  meat  of  ripe  pezu* 
exiting  ripe  pear 
hitting  a  cork  board  with  a  dart 


We  have  characterized  the  feel  of  needle  insertion  into  tomatoes  and  pears  using  the  force  and  displacement 
sensored  needle.  Figure  4  shows  the  force  readings  as  a  function  of  needle  insertion  depth  for  a  layered  set  of 
pear  and  tomato  slices.  The  outer  tomato  slice  had  an  intact  skin  in  place.  The  force  profile  of  the  pear  is  visibly 
different  from  the  force  profile  of  the  tomato  as  seen  in  Figure  4.  The  higher  forces  in  the  first  5  mm  of  tomato 
are  due  to  the  tomato  skin. 


We  have  also  used  this  data  in  lookup  table  scheme.  In  order  to  create  the  lookup  table,  the  data  of  figure  4 
was  resampled  to  a  function,  filtered,  and  scaled.  Figure  5  shows  a  plot  of  the  resulting  profile  which  can  now  be 
used  as  a  lookup  table. 

Figure  6  shows  the  output  force  as  recorded  by  the  force  sensor  during  a  needle  insertion  through  the  virtual 
pear/tomato  slices  rendered  with  the  lookup  table  of  Figure  5. 
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Figure  4:  Force/displacement  Measurements  of  Real  Layered  Tomato  and  Pear  Slices 


Figure  5:  Layered  Pear  and  Tomato  Profile:  Lookup  Table  for  Simulation 
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Figure  6:  Force/displacement  Measurement  of  Virtual  Layered  Pear  and  Tomato  Slices 


Note  first  that  the  forces  generated  in  the  simulation,  Figure  6  are  scaled  down  significantly  from  the  forces 
measured  for  generation  of  the  lookup  table.  Figure  4.  This  is  due  to  the  particular  scaling  chosen  for  the  profile, 
and  is  not  necessary,  but  does  relax  requirements  on  the  simulator-  The  shape  of  the  simulated  force  profile  is 
indeed  very  similar  to  the  shape  for  the  lookup  table.  The  motions  made  by  the  user  during  simulation  were 
similar  to  the  motions  made  during  the  recording  of  Figure  4,  that  is,  the  same  insertion  velocities  were  used. 
Had  this  not  been  the  case,  the  virtual  and  real  force  profiles  would  have  differed  a  great  deal  more.  Such  is 
the  shortcoming  of  this  lookup  table  approach.  This  can  only  be  viewed  as  a  first-cut  simulation  attempt.  Once 
again,  the  forces  felt  were  spring  forces  rather  than  damping  forces  since  position  not  velocity  was  used  to  index 
the  table. 


An  enhanced  model 


As  noted  above,  the  use  of  table  look-up  for  the  control  law  is  not  able  to  produce  a  truly  interactive  simulation. 
A  more  complete  mechanical  model  is  needed:  one  that  represents  the  driving-point  impedance  of  the  handle 
during  needle  interaction  with  tissue.  Input  to  such  a  model  would  include  velocity  and  perhaps  acceleration  of 
the  handle  in  the  user’s  grip  as  well  as  position.  An  impedance  characterization  of  the  needle/tissue  mechanics 
similar  to  the  kind  of  modeling  of  the  impedance  of  the  human  hand  underway  at  Harvard^  would  provide  models 
useable  for  fully  interactive  simulation.  A  characterization  of  the  needle’s  insertion  through  human  tissue  will 
indeed  be  a  part  of  this  project.  Data  will  be  collected  from  a  cadaver  with  an  instnimented  needle  by  medical 
instructors.  But  even  before  such  models  are  available,  we  see  promise  in  perceptual  modeling.  We  are  building 
models  (control  laws)  which  on  the  surface  appear  to  be  physical  models,  but  which  are  in  fact  perceptual 
models.  The  basic  form  contains  spring  and  damping  terms.  But  instead  of  basing  the  parameter  values  on  tissue 
cheu-acterization  experiments,  we  are  selecting  them  by  trial  2ind  error  with  human  subjects  comparing  the  virtual 
feel  to  the  actual  feel  of  needle  insertion  into  pears  and  tomatoes.  A  Windows  environment  supports  real-time 
adjustment  of  the  feel-goveming  paurameters  by  the  subjects.  Figure  7  shows  a  typical  dialog  box  with  sliders 
for  run-time  parameter  adjustment.  To  simulate  layered  materials  of  varying  mechanical  properties,  there  are 
multiple  control  laws.  Each  control  law  and  its  associated  dialog  box  has  a  region  of  applicability  or  a  pertinent 
range  of  insertion  depth  which  can  also  be  edited  during  run-time.  By  asking  subjects,  we  can  come  up  with 
model  parameters  which  minimize  the  distance  between  the  feel  of  the  virtual  and  real  tissues.  Alternatively, 
the  fit  parameter  could  be  closeness  of  the  force  measurements  made  during  interaction  with  virtual  tissue  and 
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Figure  7:  Dialog  Box 


sample  tissue. 

Other  aspects  of  the  insertion  feel  besides  its  springiness  and  damping  which  would  be  worthy  of  modeling  have 
been  suggested  by  this  heuristic  approach.  The  graininess  of  the  pear,  for  example,  appears  to  be  an  extractable 
perceptual  feature.  Sharpness  of  the  layer  interfaces  is  another.  We  are  experimenting  with  various  ad-hoc  control 
algorithms  aimed  at  simulating  individual  features  which  can  be  overlaid  or  added  to  pre-existing  models.  We 
foresee  work  along  the  lines  of  that  of  Rosenberg  and  Adlestein®  which  would  decompose  the  perception  of  the 
needle/tissue  interaction  into  a  number  of  separable  (not  necessarily  independent)  percepts. 

Finally,  the  model  can  be  can  be  further  enhanced  with  information  available  from  CT-scan.  The  model 
can  draw  its  topological  or  geometrical  description  from  the  CT-scan  data  yet  keep  its  lumped  parameter  and 
overlaid  feature  form  of  the  perceptual  model  described  above.  The  variations  seen  from  patient  to  patient  which 
are  avedlable  from  the  CT-scan  would  be  a  very  valuable  addition.  The  tissues  would  each  be  identified  and  their 
geometrical  form  maintained,  but  the  simulation  of  their  interaction  with  a  needle  would  be  governed  by  other 
models. 


Summary 


A  number  of  models  have  been  presented  each  of  which  might  foim  the  basis  of  an  epidural  analgesia  simulator. 
Data  from  a  force  sensor  and  from  the  reports  of  users  has  been  useful  for  the  generation  of  models  and  criticism 
of  models.  Since  our  aim  is  to  inspire  perceptions  in  the  users  of  the  epidural  simulator  product,  and  also  because 
adequate  physical  models  are  as  yet  unavailable,  perceptual  modeling  has  come  to  the  fore  as  the  primary  model 
development  tool. 
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This  paper  presents  an  overview  of  an  ongoing  collaborative  effort  between  researchers  at  The 
Ohio  State  University  Hospitals,  Immersion  Corporation  ,  and  the  Ohio  Supercomputer  Center  to 
create  and  test  a  virtual  simulator  for  training  residents  in  the  use  of  regional  anesthesiology. 
Specific  issues  and  difficulties  of  the  epidural  technique  are  used  to  develop  a  pilot  system.  We 
present  our  design  and  methodology,  as  well  as  considerations  for  system  integration  and  design 
iteration.  By  providing  a  new  form  of  procedural  training  in  a  nonthreatening  environment,  the 
simulator  will  increase  the  proficiency  level  of  the  resident  in  technique  delivery  and  improve 
competency  required  for  live  human  trials. 

INTRODUCTION 

Epidural  analgesia  is  one  of  the  most  frequently  used  techniques  for  the  relief  of  pain  during 
surgery.  In  most  hospitals  with  an  obstetric  anesthesia  service,  epidural  analgesia  is  the  most 
prominently  used  anesthesia  techniqtie  for  vaginal  childbirth  and  cesarean  section.  The  procedure 
involves  the  injection  of  a  local  anesthetic  or  opiod  into  the  epidural  .space  of  the  spinal  column. 
Although  a  single  degree-of-freedom  task,  it  is  a  delicate  manual  operation  that  requires  the 
placement  of  a  catheter  into  the  epidural  space  using  only  haptic  cues  to  guide  the  needle.  By 
feeling  the  resistive  forces  of  the  needle  passing  through  the  various  tissues,  the  anesthesiologist 
must  maneuver  the  tip  of  the  needle  into  the  correct  space  without  perforating  and  damaging  the 
spinal  cord  in  the  process.  Limitations  of  physical  models  such  as  mannequins  include  lack  of 
patient  variance,  inaccurate  representation  of  biological  tissue,  and  physical  wear  from  repeated 
use.  The  use  of  cadaveric  material  offers  limited  opportunities  and  associated  risks.  The  be.st 
method  of  training  residents  on  this  delicate  and  dangerous  manual  task  remains  the  use  of  live 
patients,  a  scenario  obviously  not  optimal  for  patients.  In  addition,  teaching  this  technique 
requires  highly  intensive -tutorial  interaction  with  faculty  due  to  the  significant  learning  curve  in 
understanding  exact  placement  of  the  needle. 
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INTERACTIVE  TECHNOLOGY  &  THE  NEW  PARADIGM  FOR  HEALTHCARE 

Under  funding  from  the  Department  of  Defense,  we  are  creating  a  system  for  teaching  a  specific 
method  of  regional  anesthesia,  the  epidural  technique.  Our  methods  include  the  application  and 
integration  of  virtual  technologies.  Our  system  components  include  a  high-performance  graphics 
workstation  capable  of  stereo  display,  a  real-time  volume  renderer,  a  voice  activated  interface, 
and  a  one-dimensional  haptic  probe  capable  of  simulating  the  resistive  forces  of  penetrated 
tissues. 

The  system  will  enable  the  resident  to  investigate  various  three-dimensional  reconstructed  data 
sets  in  a  non-threatening  environment.  The  system  can  be  cued  through  voice  activation  to 
provide  additional  information  in  text,  audio,  or  graphical  form.  Furthermore,  the  system 
incorporates  the  necessary  components  to  allow  the  resident  to  "feel"  the  technique  as  performed 
by  the  expert. 

BACKGROUND 

In  teaching  regional  anesthetic  technique,  several  important  issues  related  to  the  learning  curve 
have  been  ob.served: 

1)  the  process  is  labor  intensive,  necessitating  preparation  of  equipment,  and  positioning  and 
preparation  of  the  patient, 

2)  the  process  is  time  intensive  due  to  some  of  the  issues  noted  above  and  because  repeated 

exposures  to  patient  variance  must  be  experienced  before  the  student  "learns"  the  technique 

\ 

3)  the  process  is  less  than  optimal  for  the  patient  because  anesthesia  is  being  administered  by 
individuals  with  entry  level  skills.  Misplacement  of  the  needle  will  cause  pain  and  result  in 
failure  or  delay  of  pain  relief. 

In  an  effort  to  develop  a  simulator  that  maximizes  transfer  to  real  world  practice,  our  design 
considered  the  following  goals: 

1)  the  system  must  be  able  to  integrate  multisensory  information,  most  imponantly,  haptic 
display 

2)  the  system  should  employ  an  intuitive  interface  that  suppons  and  promotes  interacrion 

3)  the  system  must  support  the  presentation  of  patience  variance. 

TASK  DESCRIPTION 

The  successful  delivery  of  an  epidural  technique  is  a  demanding  task.  It  requires  the  integration 
of  limited  perceptual  cues  with  stored  knowledge.  Through  direct  palpation,  the  resident  locates 
the  proper  landmarks.  For  obstetric  analgesia,  the  second,  third,  and  fourth  lumbar  interspace  is 
chosen  because  the  epidural  space  is  largest  at  these  levels.  In  some  cases,  for  example  morbidly 
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obese  patients,  this  task  may  not  be  possible,  thereby  further  limiting  the  cues  necessary  for 
proper  orientation  and  thus  increasing  the  risk  of  misplacement  of  the  needle.  After  inserting  the 
needle,  the  anesthesiologist  must  correlate  haptic  cues,  predominantly  resistance,  with  mental 
imagery  of  the  regional  anatomy.  In  haptic  perception  (similar  to  visual  perception)  rapid 
changes  in  gradient  are  most  significant.  During  the  epidural  procedure,  resistive  forces  are  most 
apparent  at  regions  between  structures,  .such  as  facial  planes  between  muscles  or  interfaces 
between  muscle  and  ligaments  and  between  ligaments  and  bones.  The  anesthesiologist  must  use 
the  significance  of  these  changes  in  resistive  forces  to  properly  identify  the  location  of  the  needle 
within  the  patient. 

Upon  successful  placement  of  the  needle  in  the  epidural  space  surrounding  the  spinal  cord,  the 
inner  sleeve  of  the  needle  is  removed  and  a  catheter  is  inserted.  The  catheter  is  connected  to  an 
infusion  pump,  which  the  anesthesiologist  controls  during  the  course  of  delivery  or  surgical 
operation. 

There  are  two  common  techniques  for  the  administration  of  an  epidural.  One  is  the  median 
approach,  by  which  the  needle  is  placed  between  the  interspaces  of  the  lumbar  spinous  processes, 
and  the  trajectory  of  the  needle  is  in  the  midsagital  plane.  The  second  approach  is  the 
paramedian.  Here  the  needle  is  inserted  lateral  to  the  midline.  To  begin  our  development,  we 
have  chosen  the  median  approach  for  a  vinual  simulator  for  training  residents  (see  Figure  1). 


Figure  1  Figure  la  (left)  shows  trajectory  of  needle  for  median  approach.  (Adapted  from  Gray’s) 
Figure  lb  (right)  shows  trajectory  through  magnetic  re-sonance  image. 
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SYSTEM  DESCRIPTION 

Current  development  is  on  a  Silicon  Graphics  CRIMSON^'^  VGXT  graphics  workstation  (see 
Figure  2).  It  is  assumed  that  residents  are  naive  to  the  UNIXt*^  environment.  The  program  is 
evoked  through  the  use  of  speech  software,  specifically  discrete  phrase  recognition.  The  system 
prompts  the  residents  whether  the  patient  information  should  be  in  either  text  or  audio  format. 
Upon  receiving  an  answer,  the  system  provides  the  information.  The  monitor  displays  an  image 
of  the  back  of  the  patient  in  the  prepared  position.  The  current  representation  of  the  lower  back 
data  is  a  volumetric  reconstruction  from  isotropic  magnetic  resonance  data  acquired  by 
employing  a  3D  spoiled  gradient  echo  protocol  and  a  custom  RF  coil  placed  directly  on  the 
surface  of  the  back  (see  figure  lb  /  figure  3).  As  computed  tomography  would  subject  an 
individual  to  too  high  a  dose  of  radiation,  a  unembalmed  cadaver  will  be  used  to  obtain  magnetic 
resonance  and  computed  tomography  data  sets,  which  will  be  merged.  In  addition, 
instrumentation  retrofitted  with  strain  gauges  will  be  used  to  sample  forces  from  techniques 
performed  on  the  cadavar.  Current  investigations  involve  both  physical  and  perceptual  modeling 
of  the  resistive  forces. 


Figure  2:  Concept  sketch  of  system 
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The  entire  visual  presentation  is  in  stereo  to  facilitate  depth  perception.  Residents  wear  a  set  of 
stereo  viewing  glasses  that  are  synchronized  with  the  monitor  through  an  infrared  emitter,  A 
vinual  referent  of  an  epidural  needle  floats  in  front  of  the  representation  of  the  back.  A  haptic 
probe  is  positioned  in  front  of  the  monitor.  As  residents  push  the  needle  in  the  probe,  the  virtual 
needle  moves  correspondingly.  Cast  shadows  of  the  needle  are  rendered  to  cue  the  residents  as  to 
the  proximity  of  the  tip  of  the  needle  to  the  surface  of  the  back.  As  the  needle  enters  the  virtual 
back,  residents  can  feel  the  resistive  forces  of  the  underlying  tissues. 

At  any  time  during  the  procedure,  residents  who  wish  to  augment  their  mental  map  of  the 
regional  anatomy  can  request  additional  visual  information  from  the  system  through  the  speech 
interface.  Specifically,  residents  can  request  that  the  system  present  a  section  of  the  regional 
anatomy  containing  the  plane  of  the  needle.  Through  stored  voice  files,  the  system  prompts  with 
options  to  view  either  axial,  coronal,  .sagital,  or  all  three  images.  The  images  show  the  intensity 
values  from  the  magnetic  re.sonance  data  set  with  an  overlay  of  the  needle  (similar  to  Figure  lb). 
However,  during  section  presentation,  residents  are  not  permitted  to  move  the  needle  in  the 
probe.  This  is  to  prevent  dependency  on  the  visual  display  to  accomplish  the  technique.  After 
identifying  the  location  of  the  needle,  residents  can  orally  request'to  resume  the  session, 
whereupon  the  images  of  the  .sections  are  removed  and  the  display  returns  to  tlfe  exterior  view  of 
the  back  with  the  needle.  Residents  can  proceed  with  placement  of  the  needle  or  may  terminate 
the  session  through  voice  command. 

DISCUSSION 

A  limitation  of  the  current  system  is  that  it  does  not  allow  for  random  position  of  the  needle.  This 
limitation  is  why  the  median  approach  was  chosen.  With  further  hardware  advances,  further 
degrees  of  freedom  will  be  available,  and  the  resident  will  be  able  to  place  the  needle  on  the 
surface  of  the  back.  We  will  then  expand  the  system  to  simulate  the  paramedian  approach. 

In  the  development  ot  the  .system,  we  gave  some  consideration  to  using  a  physical  model  of  the 
back  with  the  force  reflection  hardware  contained  within.  Various  data  sets  of  the  internal 
anatomy  could  then  be  presented  in  the  simulation.  However,  there  are  two  reasons  for  selecting 
our  current  method. 

First,  a  physical  model  would  pre.sent  a  single  source  of  surface  anatomy  and  would  limit  the 
patient  variance  presented.  Through  volumetric  acqui.sition,  multiple  data  sets  can  be  acquired 
and  both  internal  and  external  anatomy  can  be  experienced,  thereby  maximizing  the  transfer  to 
actual  practice.  In  addition,  future  versions  will  include  a  methodology  for  palpating  the  surface 
of  the  vinual  data,  as  this  is  the  initial  and  best  cue  for  proper  orientation  and  needle  placement. 
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Second,  if  a  model  was  used,  the  display  of  sections  would  need  to  be  placed  on  a  separate 
monitor.  It  is  advantageous  to  provide  this  information  in  the  viewing  port  that  contains  the  field 
of  view  for  performing  the  technique.  Providing  the  representation  of  the  bac'k  with  internal 
representations  in  the  same  view  serves  to  limit  head  movement  and  provides  focus  for  the 
residents. 

When  requesting  the  sections  demonstrating  the  location  of  the  needle,  it  would  be  possible  to 
rotate  and,  through  constructive  solid  geometry,  expose  the  plane  to  the  viewer  (see  Figure  3). 
This  may  be  helpful  for  allowing  the  residents  to  orient  themselves  to  the  sectioned  plane.  It 
could  be  an  option  that  is  tested  for  its  efficacy.  However,  this  may  be  unnecessary  for 
orientation  and  could  prove  di.sconcerting  to  the  residents.  We  feel  the  current  method  is  more 
direct  and  straightforward. 


Figure  3:  Selected  frames  from  simulation  using  volumetric  rccon.siruction  of  magnetic  re.sonnance  images. 

Most  important,  the  presentation  of  sections  is  the  only  method  to  inform  residents  of  a 
misplaced  needle.  It  may  be  beneficial  to  incorporate  movement  of  the  data  set  along  with 
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audible  patient  responses,  i.e.,  sighs,  moans  and  an  occasional  scream.  These  would  be 
associated  with  vital  structures  in  the  data  set.  As  patient  response  is  the  prime  cue  for  improper 
placement  of  the  needle,  this  method  might  be  best  to  maximize  transfer. 

An  additional  value  of  the  simulator  is  that  it  can  be  used  on  demand.  We  envision  the  residents 
room  equipped  with  a  simulator  that  can  be  readily  accessed.  The  simulator  can  track  and 
evaluate  proficiency  levels  for  the  instructor.  During  direct  instructor  supervision,  a  master/slave 
scenario  incorporating  two  probes  can  be  set  up  to  allow  residents  to  experience  and  "feel"  the 
,  technique  as  delivered  by  the  expert,  or  the  instructor  can  feel  the  forces  exerted  by  the  residents. 

Evaluation  of  the  system  will  involve  expert  evaluation  by  faculty  and  testing  by  the  residents  in 
the  Department  of  Anesthesiology.  Graphics  and  interface  specialists  at  Immersion  Corporation 
and  the  Ohio  Supercomputer  Center  will  examine  the  technical  efficacy  of  the  virtual 
simulation/interface.  Measures  and  standards  for  judging  successive  operations  will  be  defined, 
and  all  technical  development  will  be  concurrently  validated  for  applicability. 
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APPENDIX  E:  Details  of  Electronics  Design 


We  have  developed  an  electronics  architecture  which  allows  a  standard  PC  computer  to  interface 
with  our  force  feedback  hardware.  This  architecture,  shown  in  Figure  EE-1,  can  be  broken  up 
into  a  number  of  subsystems  which  include  an  ISA  CARD,  a  POWER  AMPLIFIER,  and  the 
SENSOR/ACTUATOR  subsystem.  The  ISA  CARD  is  the  central  component  which  allows  the 
other  subsystems  to  interface  with  the  mother  board  of  the  host  computer.  The  POWER 
AMPLIFIER  provides  power  to  the  actuators,  based  on  the  low  level  force  command  signals 
produced  by  the  ISA  card.  These  power  amplifiers  use  their  own  power  supply  to  remain 
independent  of  the  host  computer  and  to  provide  maximum  performance.  Finally,  the  sensors 
and  actuators  in  the  SENSOR/ACTUATOR  subsystem  provide  the  position  input  and  force 
output  respectively  for  the  user  interface.  The  following  section  provides  a  detailed  description 
of  each  of  these  subsystems. 


Figure  EE-1,  Flow  Diagram  of  Primary  Electronics  Systems 


ISA  Interface  Card 

The  ISA  Interface  Card  provides  an  industry  standard  means  of  including  our  force  feedback 
hardware  in  a  personal  computer  system.  The  card  fits  into  an  ISA  slot  in  standard  AT 
compatible  personal  computers,  typically  based  on  Intel  and  Microsoft  DOS/Windows 
platforms.  As  shown  in  figure  EE-2,  the  ISA  card  contains  a  number  of  internal  components. 
These  include  the  buffering  logic,  the  decoding  logic,  the  circuit  banks,  and  the  fused  power 
supply. 
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Figure  EE-2,  ISA  Card  Overview  Schematic 


The  buffering  logic  relays  signals  from  the  ISA  Bus  to  the  chip  components,  minimizing  current 
draw  from  the  bus  logic  lines.  The  decode  logic  selects  which  bank  (to  be  described  later)  is 
activated  in  the  circuitry.  The  decode  logic  can  support  up  to  64  banks.  These  banks  can  each 
support  16  2-byte  (up  to  16  bit)  devices.  The  16  devices  per  bank  can  consist  of  up  to  8  input 
devices  and  up  to  8  output  devices.  The  user  uses  the  card  by  reading  from  or  writing  to  one  of 
several  of  these  devices. 


The  PC  addresses  these  devices  using  the  following  scheme: 


Lower  1 6  Bits  of  Address 


15  14 


13  12 

11 

□□ 

B 

6 

5 
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1  3  1  2 

Bank  # 
(0-63) 


Must  match 
card's  base 
address. 


Device  # 
within 
bank  (0-7) 


MSB/LSB 
Selection 
for  chosen 
device 


The  card’s  base  address  is  selected  using  a  5  position  DIP  switch.  This  5-bit  number  is 
compared  to  A8-4  when  accessing  the  card.  Pin  A9  must  be  high  (indicating  port  I/O,  not 
memory  I/O)  in  order  to  access  the  card. 
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Banks 


While  the  ISA  card  is  designed  to  accommodate  up  to  64  banks,  the  card  which  we  are  using  for 
our  force  feedback  systems  only  uses  4  of  these. 

•  Bank  0.  Bank  0  contains  simple  read/write  registers  for  status,  modes,  and  general  purpose  digital 
I/O.  Bank  0  toggles  the  control  lines  to  the  other  banks,  providing  special  features  such  as  reset, 
clear,  set  to  value,  etc.  Bank  0  consists  primarily  of  latches  (LS273)  and  drivers  (LS541). 

•  Bank  1.  Bank  1  is  used  to  read  in  values  from  up  to  8  optical  encoders  as  used  in  the  sensing 
mechanisms  of  the  force  feedback  interface.  Optical  encoders  provide  incremental  rotary  signals 
which  are  accumulated  and  updated  in  the  HCTL-2016  quadrature  chips.  These  chips  multiply 
the  pulses  coming  from  the  encoders  by  4,  using  relative  phase  changes  between  a  2  channel  input 
stream,  to  obtain  a  angular  position  value  which  is  accurate  to  4  times  the  resolution  of  the 
optical  encoder. 

•  Bank  2.  Bank  2  serves  both  as  an  analog  output  bank  and  a  analog  input  bank.  For  output,  this 
bank  can  support  up  to  8  DACs  (digital  to  analog  converters)  and  up  to  8  ADCs  (analog  to  digital 
converters).  The  DAC  banks  are  intended  to  drive  motor  amplifiers.  Since  the  motors  should 
not  be  allowed  to  “kick”  at  full-scale  when  the  PC  powers  up,  we  have  produced  the  following 
circuit  as  shown  below  in  figure  EE-3.  On  power-up,  the  Reset  register  is  cleared,  and  the  card 
outputs  float.  If  the  user  writes  to  the  Reset  register,  individual  DACs  are  enabled/disabled  by 
closing  the  output  switch. 


Figure  EE-4.  DAC  Bank  showing  Analog  Switches  Used  to 
isolate  DACs  from  output  controller. 


•  Bank  3.  Bank  3  supports  additional  ADCs  for  future  expansion  of  devices  such  as 

accelerometers,  tachometers,  and  other  additional  feedback  devices.  The  force  feedback 
platform  has  been  deliberately  left  very  open-ended  to  allow  for  future  expansion  of  features  and 
extra  feedback. 


Power  Amplifler  Circuit 

The  outputs  of  the  DAC  devices  on  the  ISA  card  send  analog  voltages  to  the  power  amplifier 
circuit.  These  voltages  are  sent  outside  of  the  computer  to  an  external  case  containing  the  power 
amplifier  and  power  supply  circuitry.  This  external  case  is  designed  to  support  varying  numbers 
of  actuators  in  order  to  accommodate  multiple  degree  of  freedom  systems  in  a  modular  manner. 
To  add  a  degree  of  freedom  usually  involves  strategically  adding  an  actuator,  in  which  case  a 
power  amplifier  module  is  added  inside  this  external  case.  A  power  cable  then  runs  from  the 
external  case  to  the  actual  force  reflecting  hardware. 
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While  motors  and  brakes  are  often  controlled  using  pulse  width  modulation  (PWM)  of  the 
applied  voltage,  this  project  requires  linear  control  of  the  applied  current.  Voltage  PWM  directly 
controls  a  motor’s  velocity,  while  linear  current  control  directly  affects  its  torque.  Since  this 
project  requires  production  of  a  given  force,  the  driver  amplifier  is  a  linear  transconductance 
amplifier. 

The  Power  Transconductance  Amplifier  Circuit,  shown  below  in  figure  EE-5,  allows  a  low- 
power  control  voltage  to  command  a  high-power,  current-controlled  load.  This  amplification 
provides  an  interface  between  the  system’s  voltage-output  DAC  and  current-input  motors  and 
brakes.  The  input  voltage  controls  a  transconductance  stage,  composed  of  U1  and  several 
resistors.  This  first  stage  produces  an  output  current  proportional  to  the  input  voltage,  while 
drawing  very  little  current  from  the  input  voltage  source.  The  second  amplifier  stage  provides 
additional  current  capacity  by  enhancing  the  voltage  swing  of  the  motor’s  second  terminal. 


10  kn 


Power  Transconductance  Amplifier 
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Figure  EE-5.  The  Power  Transconductance  Amplifier  Circuit 
Sensors/Actuators 

Sensing  is  performed  currently  by  optical  encoders  made  by  Hewlett  Packard.  These  American- 
made  sensors  provide  resolution  down  to  0.0009”  for  our  prototype  hardware  developed  in 
Phase  I.  The  sensors  can  be  accessed  directly  using  the  HCTL-2016  quadrature  chips  on  the  ISA 
card.  Other  sensors,  such  as  accelerometers,  tachometers,  tool  handles,  and  other  analog/digital 
input  sources,  can  be  connected  directly  to  one  of  the  16  analog  input  lines  or  one  of  8  digital 
input  lines  supported  by  the  ISA  card.  We  actuate  the  Phase  I  devices  with  magnetic  particle 
brakes  and  servo  motors.  These  products  can  provide  at  least  2  pounds  of  force  to  the  user  and 
are  driven  by  the  DAC  devices  in  the  ISA  card  and  powered  by  the  power  amplifier  circuitry. 

Software 

High-level  software  has  been  written  for  the  ISA  card  system,  allowing  the  following  set  of 
commands  to  be  performed: 

•  Read  the  ADC.  Read  analog  input  values.  These  functions  allow  the 
programmer  to  check  the  ADC  value,  set  up  the  ADC  for  obtaining  a  value, 
and  verifying  that  a  valid  value  exists  in  the  ADC  register. 

•  Write  to  the  DAC.  Command  an  analog  voltage  to  an  external  device.  This  is 
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used  to  send  command  voltages  to  drive  actuators. 

•  Read  position.  Collects  data  from  quadrature  chips  to  obtain  current  position 
of  rotary  encoder. 

•  Reset  encoders.  Resets  the  quadrature  chips  to  a  predefined  position. 

•  Turn  DAC  on  and  off.  This  prevents  unnecessary  power  consumption  in  the 
DACs  and  prevents  dangerous  kicks  on  start-up/shut  down. 
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